Colab demo error

[donghoonlee20]

I get this warning from import

/content/Grounded-Segment-Anything/GroundingDINO/groundingdino/models/GroundingDINO/ms_deform_attn.py:31: UserWarning: Failed to load custom C++ ops. Running on CPU mode Only!
  warnings.warn("Failed to load custom C++ ops. Running on CPU mode Only!")
/content/Grounded-Segment-Anything/GroundingDINO/groundingdino/models/GroundingDINO/ms_deform_attn.py:31: UserWarning: Failed to load custom C++ ops. Running on CPU mode Only!
  warnings.warn("Failed to load custom C++ ops. Running on CPU mode Only!")

and when running detection, I get

/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py:907: FutureWarning: The `device` argument is deprecated and will be removed in v5 of Transformers.
  warnings.warn(
/usr/local/lib/python3.10/dist-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
  warnings.warn(
/usr/local/lib/python3.10/dist-packages/torch/utils/checkpoint.py:61: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn(
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
[<ipython-input-10-eb2d1c4b4783>](https://localhost:8080/#) in <cell line: 1>()
----> 1 annotated_frame, detected_boxes = detect(image, text_prompt="bench", model=groundingdino_model)
      2 Image.fromarray(annotated_frame)

24 frames
[/content/Grounded-Segment-Anything/GroundingDINO/groundingdino/models/GroundingDINO/ms_deform_attn.py](https://localhost:8080/#) in forward(ctx, value, value_spatial_shapes, value_level_start_index, sampling_locations, attention_weights, im2col_step)
     51     ):
     52         ctx.im2col_step = im2col_step
---> 53         output = _C.ms_deform_attn_forward(
     54             value,
     55             value_spatial_shapes,

NameError: name '_C' is not defined

[merveenoyan]

I think this is an error related to whatever's running behind Colab as of now. I tried the version of GroundingSAM that was two weeks ago and still got the error. Moreover, this repository always installs the latest stable version of everything, it'd be nice to always pin the dependencies to proven working versions of each dependency (so can't guarantee this is only a Colab error until I try on another CUDA device which I can't rn as I tried with GCP and there was no T4 instance).

Anyhow, it would be great if someone would point out how to solve this. I tried a bunch of things but didn't work. torch version: 2.1.0+cu121

[maxpolaczuk]

Hi @donghoonlee20 @merveenoyan I quickly put together a colab today for one of the new efficient models that runs on T4 (because the other notebooks are like a year old and not updated). You can access it here, the readme of that repo has the "open in colab" button on it.

Hope it helps you.

https://github.com/maxpolaczuk/ai-nocode-notebooks/blob/main/notebooks/02_grounded_sam.ipynb