[REQ] add GigaGAN - Githubissues

AmusementClub / vs-mlrt

Efficient CPU/GPU ML Runtimes for VapourSynth (with built-in support for waifu2x, DPIR, RealESRGANv2/v3, Real-CUGAN, RIFE, SCUNet and more!)

GNU General Public License v3.0

310 stars 20 forks source link

[REQ] add GigaGAN #111

Open MarcoRavich opened 1 month ago

MarcoRavich commented 1 month ago

Hi there, thanks for your work !

It would be great to have GigaGan upscaling too: https://mingukkang.github.io/GigaGAN/

Some - maybe useful - implementations:

@lucidrains's GigaGAN - Pytorch;
@JiauZhang's GigaGAN;
@jianzhnie's GigaGAN.

Hope that inspires !

note: unfortunally VideoGigaGAN - by Adobe Research - sources are not (yet ?) availabe...

WolframRhodium commented 1 month ago

Hi!

One problem with these existing alternative implementations is that they may perform worse than the original. In fact, it seems that none of these provides pre-trained weights.

The second problem is that the GigaGAN model uses text conditioning, which is hard to apply to video in general I think.

NineMeowICT commented 1 month ago

@WolframRhodium An available weight by FAL: https://blog.fal.ai/introducing-aurasr-an-open-reproduction-of-the-gigagan-upscaler-2/ When it comes to super resolution, it doesn't require text conditioning. You can refer to this: https://github.com/GreenLandisaLie/AuraSR-ComfyUI

WolframRhodium commented 1 month ago

Thanks for the information.

The model is interesting because I need to improve existing vs-mlrt infrastructures to support it. This will take time.