AutoModelForZeroShotObjectDetection does not work

[stupidyoh]

Hello, First of all, I would like to thank you for your excellent research.

I am currently testing the "grounded_sam2_tracking_demo_custom_video_input_gd1.0_hf_model.py" And it seems that the issue arises from the line grounding_model = AutoModelForZeroShotObjectDetection.from_pretrained(model_id).to(device) as I am unable to load the model from Hugging Face.

Could you please let me know if the model is currently unavailable? Additionally, is there a way to set up the grounding_model in the same manner as in the local demo within this Python file?

Thank you for your assistance!

[rentainhe]

Hello @stupidyoh , the model is available in huggingface now, would you like to check if it is network issue in your local env or not. If there're some network issue, you can use another mirror source:

export HF_ENDPOINT=https://hf-mirror.com

Additionally, is there a way to set up the grounding_model in the same manner as in the local demo within this Python file?

Yes it is the same, and we will try to provide this code this week as soon as possible. Stay tuned, you can also try to implement it yourself referring to this code: https://github.com/IDEA-Research/Grounded-SAM-2/blob/main/grounded_sam2_local_demo.py

[rentainhe]

We've implemented local model for custom video tracking in our repo, you can try this demo as follows:

python grounded_sam2_tracking_demo_custom_video_input_gd1.0_local_model.py

[stupidyoh]

I was modifying the local_demo because I couldn't resolve the network issue. Thank you for uploading the new code. I have confirmed that there are no issues with the running the code.

[rentainhe]

Hi @stupidyoh , you're welcome, you can try to implement your idea based on our code, if there is any other problem you can raise another issue.