search

Jeff-LiangF / streamv2v

Official Pytorch implementation of StreamV2V.
https://jeff-liangf.github.io/projects/streamv2v/
Other
429 stars 51 forks source link

ControlNet support? #8

Closed moonsunstars39 closed 2 months ago

moonsunstars39 commented 2 months ago

Hello, First, thank you for building this great project :)

I have a quesiton here, is that possible to add ControlNet into this project in order to improve the output quality? The output video is a kind of unstable right now.

Thank you in advance :)

Jeff-LiangF commented 2 months ago

Hi @moonsunstars39 ,

I do agree that ControlNet would bring more consistency to the output. However, we don't have any plan to support ControlNet as of now. If you are interested, you are welcomed to do so by making a pull request.

Besides the effort to integrate the ControlNet-LCM into the code, the key should be swaping the attn_processors with our own cached_attn_processors if we want to use new components (EA, FF, DyMe).

Let me know if you have any questions.