search

lucidrains / imagen-pytorch

Implementation of Imagen, Google's Text-to-Image Neural Network, in Pytorch
MIT License
8.07k stars 766 forks source link

Seek for advice to reproduce Gen-1 #333

Open liuquande opened 1 year ago

liuquande commented 1 year ago

Dear author,

Thanks for your great efforts maintaining this project. I have been working on 2D image generation with diffusion for half year, and now would like to extend to video generation. I would like to reproduce the results of gen-1 (https://research.runwayml.com/gen1), i.e., transferring video content and style of given video. Since I'm relatively new to video generation, could you please give me some advice which repo should I start to train or fine-tune such a task.

Many thanks, Best.

varunponda commented 1 year ago

Hi @liuquande, If you want to generate videos from text, @lucidrains has himself implemented phenaki. If you want to generate long form videos (upto 2 mins) you can use phenaki but it is not as realistic as its contemporary. if want realistic video but you are willing to trade of duration you can check out Make-A-Video by meta AI, which again is implemented by @lucidrains. Google has also come up with imagen-video. Hope this helps.