Pdiscoformer on Hugging Face

[merveenoyan]

Hello @ananthu-aniraj 🤗

I'm Merve and I work on computer vision at Hugging Face. I see your paper is accepted as ECCV Oral, congratulations! I indexed it here https://huggingface.co/papers/2407.04538 at paper pages. The paper page lets people discuss about your paper and lets them find artifacts about it (your model for instance) you can also claim the paper as yours which will show up on your public profile at HF.

Would you like to host the model you've pre-trained with this technique on https://huggingface.co/models? I see you're using GH releases for it. Hosting on Hugging Face will give you more visibility.

If you're down, leaving a guide here. If it's a PyTorch model, you can use PyTorchModelHubMixin class which adds from_pretrained and push_to_hub to the model which lets you to upload the model and people to download and use models right away. If you do not want this and directly want to upload model through UI or however you want, people can also use hf_hub_download.

After uploaded, we can also link the models to the paper page (read here) so people can discover your model. You can also build a demo to your model on Spaces we can provide you an A100 grant.

What do you think?

[ananthu-aniraj]

Hi @merveenoyan, thanks for the information and for indexing the paper :) I can take a look at the guides. Currently, I use a hubconf.py file to load the Github-release hosted models.

[NielsRogge]

Let me know if you need any help, I can send a PR :)

Here we could indeed leverage https://huggingface.co/docs/huggingface_hub/package_reference/mixins#huggingface_hub.PyTorchModelHubMixin. We can then also add tags like image-classification or image-feature-extraction to the model cards

[ananthu-aniraj]

Hi @NielsRogge , a PR would be helpful, thanks a lot!

[ananthu-aniraj]

Hi @NielsRogge , @merveenoyan , I added the following commit to add support for PyTorchHubMixin: https://github.com/ananthu-aniraj/pdiscoformer/commit/bcd8f1433972094bfd0717c08051a02ddbd94780

Do I just need to upload the required models to my huggingface account now?

Please let me know if you get the time :)

[ananthu-aniraj]

The huggingface model loading seems to work now. (Bit hacky solution: https://github.com/ananthu-aniraj/pdiscoformer/commit/f32f6e53892a95c63f18e52d5ebb258f7aff743a)

I will upload the remaining models when I get the time.

@merveenoyan, Is there any process to follow to get an A100 grant for creating a HuggingFace Space? I will need a GPU to run the model.

[NielsRogge]

Great! Yes, after inheriting from the PyTorchModelHubMixin class, you can call model.push_to_hub("your-hf-username-or-org/pdiscoformer") for instance (make sure to log in with your HF account, using huggingface-cli login).

There's actually no need normally to leverage hf_hub_download, as you will inherit from_pretrained right away.

Regarding a grant, I'll ping the team!

[ananthu-aniraj]

All the models are now available on the Hub! I've added instructions on the model zoo file.

Thanks for sending the instructions @NielsRogge , @merveenoyan :100: Feel free to comment here if you have any updates on the grant :)

[NielsRogge]

Thanks a lot @ananthu-aniraj 🤗 cc'ing @yvrjsharma regarding the grant

[merveenoyan]

@ananthu-aniraj let me know about the link of your Space and I will assign, thanks a lot 🤗🤗

[yvrjsharma]

Hey @ananthu-aniraj, feel free to share the Space link for grants if you haven't received it already.

[ananthu-aniraj]

Hi @yvrjsharma , @merveenoyan , sorry for the delay with this ( I was on holiday after ECCV).

Here is the space: https://huggingface.co/spaces/ananthu-aniraj/pdiscoformer

It currently runs quite OK on CPU but it would be faster with GPU (I can also add batch inference in that case) :)

[NielsRogge]

Hi @ananthu-aniraj thanks for creating a demo, would be great to add some examples as it's unclear which type of images to add.

We have a guide here on how to do this with Gradio: https://www.gradio.app/guides/image-classification-in-pytorch

[ananthu-aniraj]

Thanks @NielsRogge, I'll take a look

[ananthu-aniraj]

HI @NielsRogge, for now, I've added some text instructions on expected input. Hope that workaround is fine. Otherwise I can try the gradio approach.

[yvrjsharma]

Hi @ananthu-aniraj, We offer ZeroGPU (A100s) grants for projects, but currently, they only support Gradio and not Streamlit. You may need to modify your Streamlit code to work with Gradio version 4 or 5. You can use our AI-powered playground to assist with this. It can be found here: https://www.gradio.app/playground.