search

open-mmlab / mmgeneration

MMGeneration is a powerful toolkit for generative models, based on PyTorch and MMCV.
https://mmgeneration.readthedocs.io/en/latest/
Apache License 2.0
1.87k stars 226 forks source link

Latent-Diffusion Models #483

Open Adamdad opened 1 year ago

Adamdad commented 1 year ago

Model/Dataset/Scheduler description

Recent trends show that diffusion models, especially latent diffusion models, have been very successful for image/video generation tasks. While the current codebase supports IDDPM, I believe supporting LDM for tasks like unconditional image generation or even text-to-image generation would be very helpful for the openmmlab family.

Open source status

Provide useful links for the implementation

https://github.com/CompVis/latent-diffusion

LeoXing1996 commented 1 year ago

Hey @Adamdad, thanks for your advice!

We plan to support ADM, LDM, and other diffusion models (some have already been supported but are under review 👀 ). Sampling acceleration methods such as DDIM and PNDM are in planning as well.

BTW, we have refactored MMGeneration and MMEditing based on MMEngine and MMCV 2.0, and we have recently consolidated MMGeneration into MMEditing. All the above models will be supported in MMEditing in the future. Please feel free to contact us if you want to make some contribution 🚀 !

zeakey commented 1 year ago

@LeoXing1996 Did you mean you are planning to merge mmgen to mmedit? Seems like mmedit is more frequently updated than mmgen.

LeoXing1996 commented 1 year ago

@zeakey We have already merged MMGen to MMEditing, and we are working on StableDiffusion in MMEditing. If you have any advice about the dev-1.x branch, don't hesitate to contact us.

zeakey commented 1 year ago

@LeoXing1996 Sure. I'm currently working on diffusion models and glad to see a mm-series package supports diffusion models. I will keep an eye on mmedit.

PS: Just checked out mmedit and it seems like some basic diffusion models (e.g DDPM, improved DDPM) are missing?

LeoXing1996 commented 1 year ago

@zeakey Thanks for your interest. We are working on DDPM and Improved-DDPM and will finish in early January.

lxtGH commented 1 year ago

Hey @Adamdad, thanks for your advice!

We plan to support ADM, LDM, and other diffusion models (some have already been supported but are under review 👀 ). Sampling acceleration methods such as DDIM and PNDM are in planning as well.

BTW, we have refactored MMGeneration and MMEditing based on MMEngine and MMCV 2.0, and we have recently consolidated MMGeneration into MMEditing. All the above models will be supported in MMEditing in the future. Please feel free to contact us if you want to make some contribution 🚀 !

Hi!! So it is better to use the MMEditing rather than this codebase for diffusion research? @LeoXing1996

LeoXing1996 commented 1 year ago

Hey @Adamdad, thanks for your advice! We plan to support ADM, LDM, and other diffusion models (some have already been supported but are under review 👀 ). Sampling acceleration methods such as DDIM and PNDM are in planning as well. BTW, we have refactored MMGeneration and MMEditing based on MMEngine and MMCV 2.0, and we have recently consolidated MMGeneration into MMEditing. All the above models will be supported in MMEditing in the future. Please feel free to contact us if you want to make some contribution 🚀 !

Hi!! So it is better to use the MMEditing rather than this codebase for diffusion research? @LeoXing1996

Of course, @lxtGH!!