xenova / transformers.js

State-of-the-art Machine Learning for the web. Run 🤗 Transformers directly in your browser, with no need for a server!
https://huggingface.co/docs/transformers.js
Apache License 2.0
9.92k stars 588 forks source link

What do you all think about having a "Transformers.js Community" in Hugging Face? #512

Open felladrin opened 5 months ago

felladrin commented 5 months ago

Question

After checking how MLX Community on Hugging Face is working, I thought it could be a good idea to have one for Transformers.js.

One of the key benefits of a community is "multiple curators": anyone in the community would have the ability to edit the repositories, which makes it easier to maintain the converted models and ensure that they have more detailed Readmes.

Also, having multiple curators allows for quicker resolution of issues with the model configuration. Members of the community won't need to create a pull request to request changes or wait for someone to approve the PR, which is especially important for urgent fixes.

Another good move the MLX community made was releasing a script that automatically uploads models to the organization in Hugging Face, which makes it easy for anyone to convert and share their favorite models.

I would love to hear the opinions of others.

xenova commented 4 months ago

Hi @felladrin! Sorry for the delayed response. I spoke with @osanseviero today about this, and turns out there's a lot of neat functionality built into community orgs that could really help increase models and usage (preferable to having it all under my account)!

One thing I think that would be necessary beforehand is to create a robust HF conversion space (https://github.com/xenova/transformers.js/issues/222), so users don't need to install the scripts themselves (dependency version nightmares). Do you think that's something you'd be interested in contributing?

felladrin commented 3 months ago

That's great news! I'm interested in contributing! Let me set up some PoC this week and give feedback here. I suppose I can create it in a space under my account and then move it to the new org later 🤔

osanseviero commented 3 months ago

I don't think the Space would be a blocker for the org tbh! People can also run the script and upload the converted model!

That said, having a Space would be :fire: :fire:

felladrin commented 3 months ago

Good news! I've got the space ready! https://huggingface.co/spaces/Felladrin/convert-to-onnx

Case 1. If the model hasn't been converted yet:

https://github.com/xenova/transformers.js/assets/418083/23d8431f-bf8e-4451-9aa9-7905afe2a658

...a few seconds later...

https://github.com/xenova/transformers.js/assets/418083/7ab5b0ca-9d08-4261-a600-2fd21ee58f94

Case 2. If the model has already been converted:

https://github.com/xenova/transformers.js/assets/418083/5e9126eb-fe85-45fb-b122-ed9bafad2708

To decide

File name pattern: Currently it'll upload to <org_name>/<model_creator_id>-<model_id>-ONNX. Example: TransformersJS/EleutherAI-pythia-14m-ONNX. Suggestions?

Default README.md content: Currently it doesn't write a readme, but we should. Also to include the library_name: "transformers.js" metadata.

xenova commented 3 months ago

Wow this is great! 😍 As for the community name, transformers.js-community is a bit long, so how about webml-community: https://huggingface.co/webml-community ?

Can change of course.

felladrin commented 3 months ago

webml-community fits well! 🚀

Now we just need to:

  1. Define some description for the community
  2. Define rules for the model names/cards
  3. Move the Space to the Org
  4. Announce 📢
xenova commented 1 month ago

Sorry for taking so long to revisit this... but I've also created https://huggingface.co/onnx-community which may fit better since we'll be uploading ONNX weights for all these models, and aligns well with:

This is actually born out of the need to another place to upload Transformers.js v3 models in a way that doesn't conflict with existing models.

Feel free to request to join 🔥

felladrin commented 1 month ago

Yeah!! Looks great!

But it seems it's not open for join-requests :D

image

Alternatively, you can invite me directly!

felladrin commented 1 month ago

By the way, Xenova, have you thought about the name format you're planning to use for the models uploaded to ONNX Community?

onnx-community/<model_creator_id>-<model_id>-ONNX? (e.g. onnx-community/EleutherAI-pythia-14m-ONNX)