Host model on Hugging Face

[NielsRogge]

Hi folks,

Congrats on this great work! Would you be up for making your models available on the Hugging Face hub rather than Google Drive? This would make it easier available and discoverable for the community. Here's how to do that: https://huggingface.co/docs/hub/models-uploading.

Moreover, the models can be linked to the paper, see here on how to do that.

Let me know if you need any help!

Kind regards,

Niels ML Engineer @ HF

[frank-xwang]

Hi Niels, sounds like a good idea! Thank you! Will work on it after finishing the work on hand. :-)

[NielsRogge]

Awesome! Btw I noticed that the whole image segmentation models are Mask2Former checkpoints without any changes, correct?

In that case, they could be converted to the Hugging Face Transformers format so that people can use them using that API. The conversion script to convert the weights can be found here: https://github.com/huggingface/transformers/blob/main/src/transformers/models/mask2former/convert_mask2former_original_pytorch_checkpoint_to_pytorch.py. They could in that case be hosted under your HF profile, so that people can do:

from transformers import AutoImageProcessor, Mask2FormerForUniversalSegmentation
from PIL import Image
import requests
import torch

# Load Mask2Former model and image processor
image_processor = AutoImageProcessor.from_pretrained("your-hf-profile/unsam-whole-segmentation")
model = Mask2FormerForUniversalSegmentation.from_pretrained(
    "your-hf-profile/unsam-whole-segmentation"
)

url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)
inputs = image_processor(image, return_tensors="pt")

# forward pass
with torch.no_grad():
    outputs = model(**inputs)