Making SEA-RAFT available on HF

[NielsRogge]

Hi @MemorySlices and team,

Niels here from the open-source team at Hugging Face. I discovered your work through the paper page: https://huggingface.co/papers/2405.14793 (feel free to claim the paper so that it appears under your HF account!). I work together with AK on improving the visibility of researchers' work on the hub.

I see the checkpoint is currently made available on Google Drive. It'd be great to make the models available on the hub, we can add tags so that people find them when filtering https://huggingface.co/models.

I recently did a similar integration with the author of NeuFlowv2, see https://github.com/neufieldrobotics/NeuFlow_v2?tab=readme-ov-file#inference-with-huggingface. The model is at https://huggingface.co/Study-is-happy/neuflow-v2.

For instance in this case, the "optical-flow-estimation" tag seems useful.

Uploading models

See here for a guide: https://huggingface.co/docs/hub/models-uploading. In case the model is a custom PyTorch model, we could probably leverage the PyTorchModelHubMixin class which adds from_pretrained and push_to_hub to the model. Alternatively, one can leverages the hf_hub_download one-liner to download a checkpoint from the hub.

Happy to send a PR!

We encourage researchers to push each model checkpoint to a separate model repository, so that things like download stats also work.

Demo as a Space

One could also create a Gradio demo. Happy to connect with the Gradio folks at HF on making this a breeze.

Let me know if you need any help regarding this!

Cheers,

Niels ML Engineer @ HF 🤗

[MemorySlices]

We have uploaded the models to Huggingface! Suggestions are appreciated under this issue.

[NielsRogge]

Thanks a lot! I opened a PR to improve the model card: https://huggingface.co/MemorySlices/Tartan-C-T-TSKH-spring540x960-M/discussions/1.

Would also be great to link HF in the "Model zoo" section :)