facebookresearch / fairseq

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.
MIT License
30.17k stars 6.37k forks source link

Support videoclip with newer version of dependencies #5404

Open SystemErrorWang opened 9 months ago

SystemErrorWang commented 9 months ago

🚀 Feature Request

I hope there could be a code base for videoclip with newer version of dependencies (such as python, pytorch, transformers, diffusers, etc)

Motivation

Recently I am working on a project trying to combine videoclip with diffusion model, but found the denpendencies for videoclip (MMPT) in this repository confilict with modern diffusion models, As the modern diffusion models require newer version of diffusers, transformers and other dependencies.

Pitch

Would you please provide support for videoclip models with newer version of dependencies such as diffusers, transformers, tokenizers, etc?

MarkChenYutian commented 6 months ago

Hi, I have adapted videoCLIP for a higher version of transformers for 4.37.2 from these two commits: