Making models available on HF

[NielsRogge]

Hi @haoqiwang,

Niels here from the open-source team at Hugging Face. I discovered your work through the paper page: https://huggingface.co/papers/2407.16826 (feel free to claim the paper so that it appears under your HF account!). I work together with AK on improving the visibility of researchers' work on the hub.

I see the checkpoint is currently made available on Google Drive, which might hurt their discoverability :( It'd be great to make the models available on the hub, we can add tags so that people find them when filtering https://huggingface.co/models.

For instance in this case, "image-feature-extraction" seems useful: https://huggingface.co/models?pipeline_tag=image-feature-extraction.

Uploading models

See here for a guide: https://huggingface.co/docs/hub/models-uploading. In case the model is a custom PyTorch model, we could probably leverage the PyTorchModelHubMixin class which adds from_pretrained and push_to_hub to the model. Alternatively, one can leverages the hf_hub_download one-liner to download a checkpoint from the hub.

We encourage researchers to push each model checkpoint to a separate model repository, so that things like download stats also work.

Demo as a Space

One could also create a Gradio demo. Happy to connect with the Gradio folks at HF on making this a breeze.

Let me know if you need any help regarding this!

Cheers,

Niels ML Engineer @ HF 🤗

[haoqiwang]

Thank you very much! I will upload this very soon.

[haoqiwang]

I have pushed the repo into huggingface https://huggingface.co/haoqiwang/sinder and will upload the ckpt soon. FYI how to claim the paper? I cannot find any button to click on. Thanks a lot!

[NielsRogge]

Hi, thanks for pushing the repo.

You can claim the paper by clicking on your name, like so:

Let me know if you need any help regarding pushing the weights to the hub.

[haoqiwang]

I uploaded the ckpt to the hub https://huggingface.co/haoqiwang/sinder/blob/main/dinov2g_sinder.pth

Could you please give suggestions how to use this git lfs file in the code, so that it can automatically downloaded, and then loaded into the model?

[NielsRogge]

Great! So I see you push the entire Github repository there, but actually just the weights is fine.

See DINOv2 as an example: https://huggingface.co/facebook/dinov2-base/tree/main.

To load the weights from the hub, there are a few options:

• the easiest is to inherit from the PyTorchModelHubMixin class, as it will give you the from_pretrained method. Note that this class only works in case your nn.Module class takes arguments in its init which are JSON serializable.
• you can use torch.hub.load_state_dict_from_url, where you point the URL to the HF URL of the weights. This is adopted by Meta's CoTracker here for instance.
• you can use hf_hub_download. This is for instance adopted by Meta's Vggfsm model, here.

Let me know if you need any help!