There has been a lot of confusion and a lot of questions raised since the weights for Stable Diffusion 3 was released last wednesday. I didn’t install it until saturday, but before that I had the opportunity to see what the community thought of it. It did not receive a warm welcome, to say the least.
Almost instantly the internet was flooded by horror images that was created using SD 3.
A lot of these SD 3 images more resembled what used to come from SD 1.0 and 1.5 before people started to fine tuning them.
The image above was created over 2 years ago using Stable Diffusion 1.5. Still today images like these aren’t that uncommon when using SD 1.5, if you don’t know how to prompt properly.
TESTING STABLE DIFFUSION 3
As far as I knew, the only way you could run SD 3 locally from day 1 was through ComfyUI which, honestly, I’d rather not. I have (tried to) use ComfyUI twice before , and I just don’t’ like it! But I had to install it once again, just to try SD 3 if for no other reason.
So a few days ago I installed it, and took the time to download nodes, extension, weight, and create a custom workflow suited for what I wanted to do.
The first tests I did was simply to find some decent settings for generating at least somewhat okay images with SD 3. The first thing I noticed when I started getting images that was starting to get acceptable was that SD 3 seems to have less racial bias than previous models. At least from my own very subjective experience the previous (and current) bias looks like this.
- Stable Diffusion 1.5 – bias towards asian women
- Stable Diffusion XL – bias towards caucasian people
- Stable Diffusion 3 – bias towards black women
These were among the first decent images I got from SD 3. If I use “Woman” in my prompt, more often than not she will turn out black. I don’t think that’s an issue though, if the ethnicity is important for the image then it’s easy to just put the ethnicity you want in the prompt. And it’s only fair, since asians and caucasians have been the preferred ethnicity before.
The second thing you’ll notice is the censorship, which I’ve mentioned before. When I tested SD 3 through API a couple of months ago, you could barely generate a woman at all. No matter how much clothes she had on.
Pretty much any time I tried to create an image of a woman, this was the result.
The censorship is still in place, but it’s not as bad as it used to be. From time to time SD 3 will generate a person with little to no clothes on, and it’s more creative when censoring now.
The current censorship seems to just remove the bad body parts and leave just smooth skin in their place. At least this looks funnier than just a blurry mess.
SDXL vs SD3
Eventually I ran some tests that were fast and easy to do, but at the same time not a very accurate representation. I simply went through a lot of my old images and used the old prompt to generate an image with SD 3, regardless of which model was used to generate the original image.
That will give you an idea of the differences and similarities between SDXL models and the SD 3 base model. But since I have used several different SDXL models over a period of time, and often used various tools to enhance the images, the comparison is not exactly fair towards SD 3. To be more fair I would have to use the SDXL model without any extra tools. I plan on doing that a little later on.
Here are some of the images I did make though.
I kept having a hard time generating people with SD 3 though.
SWARM UI FOR STABLE DIFFUSION
Right before I started writing this I found Swarm UI. It’s easy to install (just download 1 file and click on it) and even easier to use. And the best thing is that you can load any Stable Diffusion model (between SD 1.5 and SD 3) and use them all in the same UI. No more having to change UI depending on the model you want to use!
And the images turns out really nice as well.
These images are generated using the same prompt and settings as the pirate images above, and the difference is more than notable.
NON-COMMERCIAL RESEARCH COMMUNITY LICENSE
There has been some concern about how the license for Stable Diffusion is worded. For me and people like me that only use Stable Diffusion for our own amusement, the non-commercial license is enough.
By using or distributing any portion or element of the Models, Software, Software Products or Derivative Works, you agree to be bound by this Agreement.
“Agreement” means this Stable Non-Commercial Research Community License Agreement.
This Stability AI Model is licensed under the Stability AI Non-Commercial Research Community License, Copyright (c) Stability AI Ltd. All Rights Reserved.
The commercial license is a bit more complicated, and have by several people been interpreted as Stability AI owning everything that comes as a result of using their AI models. Basically, if you have a company and create a LoRa for your customer and they use that LoRa to create product catalogs for small companies and you stop paying for your license – then everything from the LoRa’s down to the last product catalog must be destroyed.
And not only are you fully responsible for anything you or your company does, you are also responsible for what your customers do – and what their customers do. And so forth.
STABILITY AI CREATOR LICENSE AGREEMENT
The full license is available here: License
Upon termination or expiration of this Agreement, all outstanding Orders for Software Products will terminate, all of Your payment obligations will become due and payable immediately, all rights and licenses granted to You hereunder will immediately cease, which means among other things that You and Your Customers must immediately cease using the Software Products and any Derivative Works and cease distributing any of Your Services that use or incorporate any Software Products or Derivative Works.
Upon termination or expiration of any Order or an Order Cancellation, all of Your Payment obligations will become due and payable immediately and, except as otherwise set forth in Section 9(a), all rights and licenses granted to You that are applicable to such Order will immediately cease, which means among other things that You and Your Customers must immediately cease using the Software Products and any Derivative Works related to such Order. Also upon termination or expiration, each party will return or destroy (or in the case of electronic information, render practically inaccessible) the Confidential Information of the other, including Stability’s Software Products and any Derivative Works.
These types of uncertainties has caused a lot of confusion, things like this:
Now we will just have to wait and see how this will play out, and if it will affect us who currently use Stable Diffusion under a non-commercial license.