facebookresearch / MobileLLM

MobileLLM Optimizing Sub-billion Parameter Language Models for On-Device Use Cases. In ICML 2024.
Other
1.16k stars 65 forks source link

Plans to release the model checkpoints on Hugging Face Hub? #1

Open Vaibhavs10 opened 4 months ago

Vaibhavs10 commented 4 months ago

Hey hey! Congratulations on the brilliant release. I'm VB, and I'm working on the open-source team at Hugging Face.

I was wondering if you already plan to release the model checkpoints trained as part of the paper. The community can learn a lot from these checkpoints, evaluate, test, and even fine-tune them for specific tasks!

Happy to help with the model checkpoints release and the hub stuff if it helps.

Thanks again for releasing the codebase; I'm hoping y'all release the checkpoints, too.

Cheers, VB

liuzechun commented 4 months ago

Hi VB,

Thanks for reaching out!

Yes, that's in our plan. We are working with the legal team to make the models publicly available. It is taking longer than we expected. We'll update you here once it is approved by the legal team.

Thanks, Zechun

Vaibhavs10 commented 4 months ago

Hi @liuzechun, That's brilliant! I'm looking forward to the checkpoints; there's a lot of community excitement about them!

Some quick doc links to help you when the legal approves them:

  1. Our recommendation is to have one model checkpoint per repository. You can find all the details related to upload here in this guide: https://huggingface.co/docs/hub/models-uploading
  2. You can use Collections to group and share all the model checkpoints together: https://huggingface.co/docs/hub/en/collections
  3. In addition to this you can also add MobileLLM as a library on the hub so that all the models trained from it can be neatly organised: https://huggingface.co/docs/hub/en/models-adding-libraries#register-your-libraries-supported-tasks-on-the-hub

Let me know if you need any help with this! I'm at vaibhav [at] hf [dot] co or just DM me on Twitter (reach_vb)

ahmadmustafaanis commented 3 months ago

waiting for it

redagavin commented 3 months ago

Is there any progress?

chenzx2 commented 3 months ago

waiting for it

ftgreat commented 2 months ago

waiting for it

Vaibhavs10 commented 2 months ago

Hey @liuzechun - Wondering if there's any update on the model weights? Do let me know if I/ Hugging Face can help out in anyway. πŸ€—

liuzechun commented 3 weeks ago

Hi @Vaibhavs10! Thanks for following up! @ahmadmustafaanis @redagavin @chenzx2 Thanks for your interest. 🌟The MobileLLM weights are now available on Hugging Face! πŸš€ ✨https://huggingface.co/collections/facebook/mobilellm-6722be18cb86c20ebe113e95

Vaibhavs10 commented 3 weeks ago

Thanks a ton for the release @liuzechun πŸ”₯ - this is brilliant!

Vaibhavs10 commented 3 weeks ago

Hey @liuzechun - I opened some quick PRs on the model repos to improve metadata, this will also automatically add code snippets to the model repos too:

https://huggingface.co/facebook/MobileLLM-125M/discussions/1 https://huggingface.co/facebook/MobileLLM-350M/discussions/2 https://huggingface.co/facebook/MobileLLM-600M/discussions/1 https://huggingface.co/facebook/MobileLLM-1B/discussions/1

Let me know if you have any questions! πŸ€—

liuzechun commented 3 weeks ago

Thank you @Vaibhavs10, just merged your pull requests!