Non-Stable Diffusion VAE for commercial usage?

[araleza]

Hi! Your project looks great, well done on the work so far.

Although I do open source work in my spare time, I also do image generation at work too. Lumina's MIT License is useful for commercial development, but I notice that the VAE used by Lumina is always Stable Diffusion (either 1.5, or SDXL). I think this means that using Lumina in a released commercial product requires obtaining a commercial license from stability.ai.

Are there are plans to train or support an MIT-licensed VAE so that the whole stack can be used without licensing difficulties, even in commercial projects?

[gaopengpjlab]

Do you have any recommendation?

[araleza]

I heard someone (I think it was maybe comfy) saying that training a VAE is pretty easy. I think the code for the stability.ai VAE itself is totally free to use, it's just the weights that need to be commercially licensed?

If that's true, then it should be possible to take a large number of latent outputs of your model just now, along with the images produced by using the Stable Diffusion VAE to produce the correct output images for these latents, and use these latent/image pairs to train a free-to-use VAE that's a license-free drop-in replacement for the stability.ai VAE.

In other words the procedure would be:

• Use the Stable Diffusion VAE to generate a large number of latents for images in your training image set
• Discard the weights for the Stable Diffusion VAE, but keep the free-to-use code for their model structure
• Retrain new VAE weights using the large number of {latent, image} pairs
• Upload that somewhere for people to use, with an MIT License

Or, maybe someone's already made a license-free VAE someplace?

[PommesPeter]

We are using stabilityai/sdxl-vae which has MIT license. Should this be legal?

[araleza]

I didn't know the SDXL VAE was MIT licensed. That's great to know, thanks @PommesPeter.