In my previous post I mentioned AI models but didn’t go into what they are and what they do. I will try to explain in a simple way what an AI model is, give some examples of different models and how they can be used.
What Is An AI Model?
The AI model can be said to be the core of AI and can be explained as an algorithm, or a program, that is trained to understand patterns, make decisions and make predictions. If you installed ComfyUI according to the instructions in my previous post, you downloaded the AI model Stable Diffusion version 1.4.
However, there are a variety of AI models, and if you have searched for AI online, you have probably seen models like Midjourney, Stable Diffusion or Dall-E when it comes to image-generating AI. When it comes to text AI, you have probably used or heard of ChatGPT, Chatsonic or Google Bard. These are all different types of AI models, have different developers behind them and are trained in different ways.
I can’t say with 100% certainty that no image-generating AI has been trained freely on the entire internet, but I personally think that would give a pretty poor result. The entire internet contains far too much information for that to be reasonable.
What Does It Mean to Train an AI Model?
Training a model means giving it a lot of data that it will learn to categorize correctly. It is important to remember that the AI we have today is not what’s called True AI, which would mean that it would have been autonomous and could learn things without any help whatsoever. True AI, or Artificial General Intelligence, does not exist today, but when we talk about AI, we usually mean Generative Artificial Intelligence.
Generative Artificial Intelligence needs some type of input of information to learn, and it works like this: what you get from your AI, is dependant on what you put into it. A prime example is Tay, which was an AI chatbot made by Microsoft. On March 23, 2016, Microsoft thought it sounded like a good idea to train Tay by letting it interact with users on Twitter. It might sound like a reasonable idea since there are constant text conversations going on between people there.
In just 16 hours, Microsoft shut down Tay, after Twitter users interacting with Tay caused the AI to write things like the ones in the image above. Microsoft should have realized that Tay would quickly become corrupted on Twitter. After all, this is Twitter we’re talking about, and anyone who has had an account there knows how infected all debate on Twitter is.
Generative AI
Image-generating AI models work in much the same way, and the images you create with an AI are a reflection of how it has been trained. For example, an AI that has been trained only on images of dogs assumes that everything is dog-related. If you asked this AI to create an image of a woman, it might create a female dog. Or nothing at all, since it wouldn’t understand the meaning of the word “woman” (unless it was trained to understand that the word “woman” means “female”).
We will test using the exact same settings, seed and prompt, on a few different models to see the difference.
I have used the following settings for all the images, and that is also the reason why the quality is not quite 100% on all of them.
When looking at the images below, try to ignore any inaccuracies in the individual images, and instead focus on the similarities. The point here is how the different AI models interpret and present the exact same data.
The very first thing you’ll probably notice is that the pictures are pretty much the same. The vast majority of the pictures show a girl facing left, with flowers in her hair and also flowers in the background. Only in the first picture is the girl facing a different direction. In addition, some pictures look realistic, others look like a painted portrait and the last picture is an Anime image.
Another thing that might stand out is the ethnicity of the girls in the pictures. All but one are white and have reddish hair, which is not surprising since we have used the same seed and settings for all the pictures. The reason why Mix9 Realistic and Anything v3 are different from the others is that Mix9 Realistic is trained on Asian people and Anything v3 is trained on Anime characters.
Choosing the Right AI Model Is Crucial
As we have seen, different AI models produce very different results depending on how they are trained, even when using the exact same settings and seed. Therefore, it is important to know in advance what kind of images you want to create, so that you can choose the right model based on that.
Can’t you just download all the models and change them as you go? Theoretically you coulddo that, but an AI model is usually quite large. I’ve downloaded 16 different models, and this already takes up quite a bit of space on my computer.
How many models are there? I actually have no idea, but it’s thousands. If each model is on average 2.5-3.0 GB in size (and each new model seem to be larger than the previous one), you can understand how unreasonable it is to download all the models. The most reasonable thing is to download a number of models that you have as base models, for example a few different Stable Diffusion models, and then use LoRA to get the desired result.
What LoRA is, and how to use it together with different models will be a separate post.
Until then, you can find a variety of AI models online that you can try out.
AI Models: Civitai
AI Models: Huggingface
Sign up for our weekly newsletter to stay in the loop for more goodies!