bloc97
commented
2 years ago I think the "k-diffusion" you are referring to is the k-lms scheduler? It is already used by default in this implementation.
About learning about diffusion models, I guess it would be hard to understand without previous experience with generative models like GANs and VAEs, also knowing about the network architecture that StableDiffusion uses helps too. Mostly if you want to edit the models themselves you would need a fair amount of theoretical knowledge about machine learning. I think the best way to learn would be taking a undergraduate course on machine learning then a few graduate courses about representation learning.
I got started by learning from courses and also a lot of self learning by reading papers. I'm not sure if there is any open courses similar to the one given at MILA (the one given there does not always have video recordings) (https://sites.google.com/view/ift-6135-h2020/lectures) but any course that resembles this one would get you 90% of the way there, and the last 10% catching up by reading the original paper on diffusion models (https://arxiv.org/abs/2006.11239), then the DALL-E 2 paper (https://arxiv.org/abs/2204.06125) and finally the StableDiffusion paper (https://arxiv.org/abs/2112.10752).
Where did you learn this stuff? There are a lot of "for dummies" level explanations about how diffusors work, and there's a fair amount of documentation out there that assumes you have am understanding of the inner workings of them, but there doesn't seem to be much in between. I would like to, for instance, add k-diffusion to this notebook, but I'm really confused as to how to get started. Is there any kind of reading material out there that I could use to familiarize myself with this stuff a bit more?