It seems that Grounded-Segment-Anything doesn't support Win-Conda env well

[stevezkw1998]

(base) C:\code\Grounded-Segment-Anything>python run_grounded_sam_demo.py
C:\code\Grounded-Segment-Anything\GroundingDINO\groundingdino\models\GroundingDINO\ms_deform_attn.py:31: UserWarning: Failed to load custom C++ ops. Running on CPU mode Only!
  warnings.warn("Failed to load custom C++ ops. Running on CPU mode Only!")
Traceback (most recent call last):
  File "C:\code\Grounded-Segment-Anything\grounded_sam_demo.py", line 139, in <module>
    model = load_model(config_file, grounded_checkpoint, device=device)
  File "C:\code\Grounded-Segment-Anything\grounded_sam_demo.py", line 39, in load_model
    args = SLConfig.fromfile(model_config_path)
  File "C:\code\Grounded-Segment-Anything\GroundingDINO\groundingdino\util\slconfig.py", line 182, in fromfile
    cfg_dict, cfg_text = SLConfig._file2dict(filename)
  File "C:\code\Grounded-Segment-Anything\GroundingDINO\groundingdino\util\slconfig.py", line 83, in _file2dict
    shutil.copyfile(filename, osp.join(temp_config_dir, temp_config_name))
  File "C:\ProgramData\Anaconda3\lib\shutil.py", line 266, in copyfile
    with open(dst, 'wb') as fdst:
PermissionError: [Errno 13] Permission denied: 'C:\\Users\\a4781\\AppData\\Local\\Temp\\tmpu081wnn5\\tmp5ej1pa3b.py'

-------------------------------------run_grounded_sam_demo.py-------------------------------------

import os

def run_grounded_sam_demo():
    os.environ['CUDA_VISIBLE_DEVICES'] = '0'

    os.system('python grounded_sam_demo.py '
            '--config GroundingDINO/groundingdino/config/GroundingDINO_SwinT_OGC.py '
            '--grounded_checkpoint checkpoints/groundingdino_swint_ogc.pth '
            '--sam_checkpoint checkpoints/sam_vit_h_4b8939.pth '
            '--input_image assets/demo1.jpg '
            '--output_dir "outputs" '
            '--box_threshold 0.3 '
            '--text_threshold 0.25 '
            '--text_prompt "bear" '
            '--device "cuda"')

run_grounded_sam_demo()

[SlongLiu]

Thanks for the issue. We are working to solve the problem on Windows.

[yizhikan]

https://github.com/IDEA-Research/GroundingDINO/issues/14 I followed this way. It worked. in ./GroundingDINO/groundingdino/util/slconfig.py add this line temp_config_file.close() after temp_config_name = osp.basename(temp_config_file.name)

[stevezkw1998]

IDEA-Research/GroundingDINO#14 I followed this way. It worked. in ./GroundingDINO/groundingdino/util/slconfig.py add this line temp_config_file.close() after temp_config_name = osp.basename(temp_config_file.name)

Hi Thank you for your solution, and your solution solved this issue. However, after I solved this issue, there is another issue come up:

(base) C:\code\Grounded-Segment-Anything>python run_grounded_sam_demo.py 
C:\code\Grounded-Segment-Anything\GroundingDINO\groundingdino\models\GroundingDINO\ms_deform_attn.py:31: UserWarning: Failed to load custom C++ ops. Running on CPU mode Only!
  warnings.warn("Failed to load custom C++ ops. Running on CPU mode Only!")
C:\ProgramData\Anaconda3\lib\site-packages\torch\functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\TensorShape.cpp:3484.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
final text_encoder_type: bert-base-uncased
Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertModel: ['cls.predictions.transform.LayerNorm.bias', 'cls.predictions.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.dense.bias', 'cls.predictions.decoder.weight', 'cls.predictions.transform.dense.weight', 'cls.seq_relationship.weight']
- This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
_IncompatibleKeys(missing_keys=[], unexpected_keys=['label_enc.weight'])
C:\ProgramData\Anaconda3\lib\site-packages\transformers\modeling_utils.py:830: FutureWarning: The `device` argument is deprecated and will be removed in v5 of Transformers.
  warnings.warn(
C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\checkpoint.py:31: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn("None of the inputs have requires_grad=True. Gradients will be None")
Traceback (most recent call last):
  File "C:\code\Grounded-Segment-Anything\grounded_sam_demo.py", line 145, in <module>
    boxes_filt, pred_phrases = get_grounding_output(
  File "C:\code\Grounded-Segment-Anything\grounded_sam_demo.py", line 57, in get_grounding_output
    outputs = model(image[None], captions=[caption])
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\code\Grounded-Segment-Anything\GroundingDINO\groundingdino\models\GroundingDINO\groundingdino.py", line 313, in forward
    hs, reference, hs_enc, ref_enc, init_box_proposal = self.transformer(
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\code\Grounded-Segment-Anything\GroundingDINO\groundingdino\models\GroundingDINO\transformer.py", line 258, in forward
    memory, memory_text = self.encoder(
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\code\Grounded-Segment-Anything\GroundingDINO\groundingdino\models\GroundingDINO\transformer.py", line 576, in forward
    output = checkpoint.checkpoint(
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\checkpoint.py", line 249, in checkpoint
    return CheckpointFunction.apply(function, preserve, *args)
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\autograd\function.py", line 506, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\checkpoint.py", line 107, in forward
    outputs = run_function(*args)
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\code\Grounded-Segment-Anything\GroundingDINO\groundingdino\models\GroundingDINO\transformer.py", line 785, in forward
    src2 = self.self_attn(
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\code\Grounded-Segment-Anything\GroundingDINO\groundingdino\models\GroundingDINO\ms_deform_attn.py", line 338, in forward
    output = MultiScaleDeformableAttnFunction.apply(
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\autograd\function.py", line 506, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
  File "C:\code\Grounded-Segment-Anything\GroundingDINO\groundingdino\models\GroundingDINO\ms_deform_attn.py", line 53, in forward
    output = _C.ms_deform_attn_forward(
NameError: name '_C' is not defined

[My local code is up to date] cc: @SlongLiu

[aallcg]

IDEA-Research/GroundingDINO#14 I followed this way. It worked. in ./GroundingDINO/groundingdino/util/slconfig.py add this line temp_config_file.close() after temp_config_name = osp.basename(temp_config_file.name)

Hi Thank you for your solution, and your solution solved this issue. However, after I solved this issue, there is another issue come up:

(base) C:\code\Grounded-Segment-Anything>python run_grounded_sam_demo.py 
C:\code\Grounded-Segment-Anything\GroundingDINO\groundingdino\models\GroundingDINO\ms_deform_attn.py:31: UserWarning: Failed to load custom C++ ops. Running on CPU mode Only!
  warnings.warn("Failed to load custom C++ ops. Running on CPU mode Only!")
C:\ProgramData\Anaconda3\lib\site-packages\torch\functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\TensorShape.cpp:3484.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
final text_encoder_type: bert-base-uncased
Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertModel: ['cls.predictions.transform.LayerNorm.bias', 'cls.predictions.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.dense.bias', 'cls.predictions.decoder.weight', 'cls.predictions.transform.dense.weight', 'cls.seq_relationship.weight']
- This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
_IncompatibleKeys(missing_keys=[], unexpected_keys=['label_enc.weight'])
C:\ProgramData\Anaconda3\lib\site-packages\transformers\modeling_utils.py:830: FutureWarning: The `device` argument is deprecated and will be removed in v5 of Transformers.
  warnings.warn(
C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\checkpoint.py:31: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn("None of the inputs have requires_grad=True. Gradients will be None")
Traceback (most recent call last):
  File "C:\code\Grounded-Segment-Anything\grounded_sam_demo.py", line 145, in <module>
    boxes_filt, pred_phrases = get_grounding_output(
  File "C:\code\Grounded-Segment-Anything\grounded_sam_demo.py", line 57, in get_grounding_output
    outputs = model(image[None], captions=[caption])
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\code\Grounded-Segment-Anything\GroundingDINO\groundingdino\models\GroundingDINO\groundingdino.py", line 313, in forward
    hs, reference, hs_enc, ref_enc, init_box_proposal = self.transformer(
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\code\Grounded-Segment-Anything\GroundingDINO\groundingdino\models\GroundingDINO\transformer.py", line 258, in forward
    memory, memory_text = self.encoder(
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\code\Grounded-Segment-Anything\GroundingDINO\groundingdino\models\GroundingDINO\transformer.py", line 576, in forward
    output = checkpoint.checkpoint(
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\checkpoint.py", line 249, in checkpoint
    return CheckpointFunction.apply(function, preserve, *args)
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\autograd\function.py", line 506, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\checkpoint.py", line 107, in forward
    outputs = run_function(*args)
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\code\Grounded-Segment-Anything\GroundingDINO\groundingdino\models\GroundingDINO\transformer.py", line 785, in forward
    src2 = self.self_attn(
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\code\Grounded-Segment-Anything\GroundingDINO\groundingdino\models\GroundingDINO\ms_deform_attn.py", line 338, in forward
    output = MultiScaleDeformableAttnFunction.apply(
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\autograd\function.py", line 506, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
  File "C:\code\Grounded-Segment-Anything\GroundingDINO\groundingdino\models\GroundingDINO\ms_deform_attn.py", line 53, in forward
    output = _C.ms_deform_attn_forward(
NameError: name '_C' is not defined

cc: @SlongLiu

+1 am seeing the same

[aallcg]

Looks like running: "python -m pip install -e GroundingDINO" on the root dir worked fix'd it for me.

[stevezkw1998]

Looks like running: "python -m pip install -e GroundingDINO" on the root dir worked fix'd it for me.

Hi @aallcg Thank you for your information, But I've tried running "python -m pip install -e GroundingDINO" on ‘(base) C:\code\Grounded-Segment-Anything>python -m pip install -e GroundingDINO’ Then ran the above again, still the same results, could you share more details on which specific dir you run the cmd?

[aallcg]

Right, I ran it in the base "Grounded-Segment-Anything" directory. After doing so, I noticed the GroundingDino/groundingdino now has a "_C.cp310-win_amd64.pyd" file which was previously not there. Here's how my dirs look:

[0xbitches]

Right, I ran it in the base "Grounded-Segment-Anything" directory. After doing so, I noticed the GroundingDino/groundingdino now has a "_C.cp310-win_amd64.pyd" file which was previously not there. Here's how my dirs look:

That's odd. I have the same file but running the gradio demo is still bugging out for me. Same error as above.

I tried manually importing _C but it just says ImportError: DLL load failed while importing _C: The specified module could not be found.

[stevezkw1998]

Thank you for your detail information, I observed the process, thers is no such file generated on my side, no matter I ran PS C:\code\Grounded-Segment-Anything> python -m pip install -e GroundingDINO or PS C:\code\Grounded-Segment-Anything\GroundingDINO> pip install -e .

[junglej]

IDEA-Research/GroundingDINO#14 I followed this way. It worked. in ./GroundingDINO/groundingdino/util/slconfig.py add this line temp_config_file.close() after temp_config_name = osp.basename(temp_config_file.name)

Hi Thank you for your solution, and your solution solved this issue. However, after I solved this issue, there is another issue come up:

(base) C:\code\Grounded-Segment-Anything>python run_grounded_sam_demo.py 
C:\code\Grounded-Segment-Anything\GroundingDINO\groundingdino\models\GroundingDINO\ms_deform_attn.py:31: UserWarning: Failed to load custom C++ ops. Running on CPU mode Only!
  warnings.warn("Failed to load custom C++ ops. Running on CPU mode Only!")
C:\ProgramData\Anaconda3\lib\site-packages\torch\functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\TensorShape.cpp:3484.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
final text_encoder_type: bert-base-uncased
Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertModel: ['cls.predictions.transform.LayerNorm.bias', 'cls.predictions.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.dense.bias', 'cls.predictions.decoder.weight', 'cls.predictions.transform.dense.weight', 'cls.seq_relationship.weight']
- This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
_IncompatibleKeys(missing_keys=[], unexpected_keys=['label_enc.weight'])
C:\ProgramData\Anaconda3\lib\site-packages\transformers\modeling_utils.py:830: FutureWarning: The `device` argument is deprecated and will be removed in v5 of Transformers.
  warnings.warn(
C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\checkpoint.py:31: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn("None of the inputs have requires_grad=True. Gradients will be None")
Traceback (most recent call last):
  File "C:\code\Grounded-Segment-Anything\grounded_sam_demo.py", line 145, in <module>
    boxes_filt, pred_phrases = get_grounding_output(
  File "C:\code\Grounded-Segment-Anything\grounded_sam_demo.py", line 57, in get_grounding_output
    outputs = model(image[None], captions=[caption])
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\code\Grounded-Segment-Anything\GroundingDINO\groundingdino\models\GroundingDINO\groundingdino.py", line 313, in forward
    hs, reference, hs_enc, ref_enc, init_box_proposal = self.transformer(
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\code\Grounded-Segment-Anything\GroundingDINO\groundingdino\models\GroundingDINO\transformer.py", line 258, in forward
    memory, memory_text = self.encoder(
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\code\Grounded-Segment-Anything\GroundingDINO\groundingdino\models\GroundingDINO\transformer.py", line 576, in forward
    output = checkpoint.checkpoint(
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\checkpoint.py", line 249, in checkpoint
    return CheckpointFunction.apply(function, preserve, *args)
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\autograd\function.py", line 506, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\checkpoint.py", line 107, in forward
    outputs = run_function(*args)
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\code\Grounded-Segment-Anything\GroundingDINO\groundingdino\models\GroundingDINO\transformer.py", line 785, in forward
    src2 = self.self_attn(
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\code\Grounded-Segment-Anything\GroundingDINO\groundingdino\models\GroundingDINO\ms_deform_attn.py", line 338, in forward
    output = MultiScaleDeformableAttnFunction.apply(
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\autograd\function.py", line 506, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
  File "C:\code\Grounded-Segment-Anything\GroundingDINO\groundingdino\models\GroundingDINO\ms_deform_attn.py", line 53, in forward
    output = _C.ms_deform_attn_forward(
NameError: name '_C' is not defined

cc: @SlongLiu

+1 am seeing the same

I have the same issue in Ubuntu

[GWLAN]

same issue, it seem that not support AMD RoCM