Improve Hugging Face discoverability, make download stats work

[NielsRogge]

Dear authors,

Thanks for this nice work! I wrote a quick PoC to showcase that you can easily have integration with the 🤗 hub so that you can automatically load the various VQ-VAE models using from_pretrained (and push them using push_to_hub), track download numbers for your models (similar to models in the Transformers library), and have nice model cards on a per-model basis. It leverages the PyTorchModelHubMixin class which allows to inherits these methods.

Usage is as follows:

from tokenizer.tokenizer_image.vq_model import VQModel

vq_model = VQModel.from_pretrained("nielsr/vq-ds16-c2i")

# encode
encoding, _, _ = vq_model.encode(...)

# decode
decoding = vq_model.decode(encoding)

The corresponding model is here for now: https://huggingface.co/nielsr/vq-ds16-c2i. We could move all checkpoints to separate repos to your organization on the hub. Ideally, each repo is a single checkpoint.

Would you be interested in this integration?

Kind regards,

Niels

[ShoufaChen]

Hi @NielsRogge , Thank you for your PR. Just wondering could we do similar thing for aoturegressive model?

[NielsRogge]

Yes it works for the GPT models too! Just pushed a script.

Could you kindly try out running python push_vae_to_hf.py and python push_gpt_to_hf.py?

You can of course replace my username ("nielsr") by your organization and push the checkpoints to your organization.

[PeizeSun]

Hi~ Thanks for your great PR. We are now reviewing and will merge it within next 24 hours.

[NielsRogge]

Thank you! Btw the scripts require huggingface-cli login to be ran first from the terminal to make sure you're authenticated with your HF account.

[jshilong]

Improve Hugging Face discoverability

[NielsRogge]

Hi @jshilong @ShoufaChen thanks for merging, however could you try out pushing the checkpoints to the hub? As otherwise the discoverability wouldn't be improved. Here's how you can run them:

huggingface-cli login
python push_gpt_to_hf.py
python push_vae_to_hf.py

[NielsRogge]

Hi @jshilong thanks for pushing the VAE, it looks like it's being downloaded more than 3k times last 30 days:

Would be cool if you could push the other models in a similar fashion