facebookresearch / LaViLa

Code release for "Learning Video Representations from Large Language Models"
MIT License
491 stars 46 forks source link

Add models/demo to Hugging Face Hub #1

Open nateraw opened 1 year ago

nateraw commented 1 year ago

Hey there, it would be really cool if the models/a demo were included in the facebook org on Hugging Face!

Also, it would be really cool to host a good demo of LaViLa on Spaces as well 😄 . I started a demo here... it's currently running on CPU, but if you're interested, we can move to facebook org and apply a GPU to the demo so more folks can try it out quickly.

Let me know if there's anything I can do to help 🤗

rohitgirdhar commented 1 year ago

This is amazing, thanks so much @nateraw!! :)

nateraw commented 1 year ago

My pleasure! The app currently has a GPU added to it for now for faster inference.

WDYT about sharing the models on the Hub? 😄 Anything I can do to help there?

zhaoyue-zephyrus commented 1 year ago

Hey @nateraw ,

Thanks for your efforts for the 🤗Spaces Demo!! I researched a bit on the huggingface demo and it seems that I need to port the model to 🤗Transformers (https://huggingface.co/docs/transformers/add_new_model) IIRC (or if there is any alternative solution, let me know). I might have the bandwidth to work on this in the next couple of weeks. I will keep you posted!

Best, Yue

nateraw commented 1 year ago

Hey there, there's actually no need to port to Transformers if that's too much of a time commitment.

All you need to do is:

  1. Create a repo for a given model (best to use 1 repo per model ckpt). Can be done via browser or programatically with huggingface_hub.create_repo
  2. Upload the model checkpoint, which you can do via browser or programatically with huggingface_hub.upload_file or huggingface_hub.upload_folder
  3. create/upload a model card (which is just a README.md file in your repo).
  4. (Optional) Add some ability to load these models from the Hub in your codebase here so folks don't have to go download them manually...something like Model.from_pretrained('username/repo-id'). Under the hood you'd use huggingface_hub.hf_hub_download.

Happy to answer any other questions you may have :)