search

hkchengrex / Tracking-Anything-with-DEVA

[ICCV 2023] Tracking Anything with Decoupled Video Segmentation
https://hkchengrex.com/Tracking-Anything-with-DEVA/
Other
1.27k stars 129 forks source link

Error: cannot import name 'register_model' from 'timm.models.registry' #10

Closed chris-hndz closed 1 year ago

chris-hndz commented 1 year ago

Hi!

I wanted to test the project, but after installing your fork of Grounding SAM and testing:

python demo/demo_gradio.py

I get the following error:

UserWarning: Failed to load custom C++ ops. Running on CPU mode Only! warnings.warn("Failed to load custom C++ ops. Running on CPU mode Only!") Traceback (most recent call last): File "...\Tracking-Anything-with-DEVA\demo\demo_gradio.py", line 16, in from deva.ext.grounding_dino import get_grounding_dino_model File "...\Tracking-Anything-with-DEVA\deva\ext\grounding_dino.py", line 17, in from deva.ext.MobileSAM.setup_mobile_sam import setup_model as setup_mobile_sam File "...\Tracking-Anything-with-DEVA\deva\ext\MobileSAM\setup_mobile_sam.py", line 3, in from deva.ext.MobileSAM.tiny_vit_sam import TinyViT File "...\Tracking-Anything-with-DEVA\deva\ext\MobileSAM\tiny_vit_sam.py", line 19, in from timm.models.registry import register_model ImportError: cannot import name 'register_model' from 'timm.models.registry' (...\Python\Python310\site-packages\timm\models\registry.py)

Could you please give me some guidance on how to solve this error?

Thank you very much in advance!

hkchengrex commented 1 year ago

Try upgrading timm?

chris-hndz commented 1 year ago

Thank you @hkchengrex for your quick response!

I have tried to update timm:

pip install --upgrade timm

And although it managed to update it, it sends me the following message:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. mivolo 0.2.0.dev0 requires timm==0.8.13.dev0, but you have timm 0.9.7 which is incompatible. Successfully installed timm-0.9.7

I tried to update mivolo, then update timm, but the same message continues.

I do not know if this will affect the operation of the project.

Anyway, despite this message I managed to run the demo in Gradio:

python demo/demo_gradio.py

And when I try to test the first example video (I downloaded the video to my computer and uploaded it locally), I get an error with this message:

..\Python\Python310\site-packages\torch\functional.py:478: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ..\aten\src\ATen\native\TensorShape.cpp:2895.) return _VF.meshgrid(tensors, kwargs) # type: ignore[attr-defined] final text_encoder_type: bert-base-uncased Configuration: {'model': './saves/DEVA-propagation.pth', 'output': None, 'save_all': False, 'amp': True, 'key_dim': 64, 'value_dim': 512, 'pix_feat_dim': 512, 'disable_long_term': False, 'max_mid_term_frames': 10, 'min_mid_term_frames': 5, 'max_long_term_elements': 10000, 'num_prototypes': 128, 'top_k': 30, 'mem_every': 5, 'chunk_size': 8, 'size': 480, 'GROUNDING_DINO_CONFIG_PATH': './saves/GroundingDINO_SwinT_OGC.py', 'GROUNDING_DINO_CHECKPOINT_PATH': './saves/groundingdino_swint_ogc.pth', 'DINO_THRESHOLD': 0.35, 'DINO_NMS_THRESHOLD': 0.8, 'SAM_ENCODER_VERSION': 'vit_h', 'SAM_CHECKPOINT_PATH': './saves/sam_vit_h_4b8939.pth', 'MOBILE_SAM_CHECKPOINT_PATH': './saves/mobile_sam.pt', 'SAM_NUM_POINTS_PER_SIDE': 64, 'SAM_NUM_POINTS_PER_BATCH': 64, 'SAM_PRED_IOU_THRESHOLD': 0.88, 'SAM_OVERLAP_THRESHOLD': 0.8, 'img_path': './example/vipseg', 'detection_every': 5, 'num_voting_frames': 3, 'temporal_setting': 'semionline', 'max_missed_detection_count': 5, 'max_num_objects': 200, 'prompt': 'people.hats.horses', 'sam_variant': 'original', 'enable_long_term': True, 'enable_long_term_count_usage': True} 0%| | 0/45 [00:00<?, ?it/s]..\lib\site-packages\transformers\modeling_utils.py:909: FutureWarning: The device argument is deprecated and will be removed in v5 of Transformers. warnings.warn( ..\Python\Python310\site-packages\torch\utils\checkpoint.py:25: UserWarning: None of the inputs have requires_grad=True. Gradients will be None warnings.warn("None of the inputs have requires_grad=True. Gradients will be None") 0%| | 0/45 [00:02<?, ?it/s] Traceback (most recent call last): File "..\Python\Python310\site-packages\gradio\routes.py", line 395, in run_predict output = await app.get_blocks().process_api( File "..\Python\Python310\site-packages\gradio\blocks.py", line 1193, in process_api result = await self.call_function( File "..\Python\Python310\site-packages\gradio\blocks.py", line 916, in call_function prediction = await anyio.to_thread.run_sync( File "..\lib\site-packages\anyio\to_thread.py", line 33, in run_sync return await get_async_backend().run_sync_in_worker_thread( File "..\lib\site-packages\anyio_backends_asyncio.py", line 2106, in run_sync_in_worker_thread return await future File "..\lib\site-packages\anyio_backends_asyncio.py", line 833, in run result = context.run(func, args) File "..\Tracking-Anything-with-DEVA\demo\demo_gradio.py", line 75, in demo_with_text process_frame_text(deva, File "..\Python\Python310\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context return func(args, kwargs) File "..\Tracking-Anything-with-DEVA\deva\ext\with_text_processor.py", line 58, in process_frame_with_text mask, segments_info = make_segmentation_with_text(cfg, image_np, gd_model, sam_model, File "..\Tracking-Anything-with-DEVA\deva\ext\with_text_processor.py", line 25, in make_segmentation_with_text mask, segments_info = segment_with_text(cfg, gd_model, sam_model, image_np, prompts, min_side) File "..\Tracking-Anything-with-DEVA\deva\ext\grounding_dino.py", line 73, in segment_with_text detections = gd_model.predict_with_classes(image=cv2.cvtColor(image, cv2.COLOR_RGB2BGR), File "..\tracking-anything-with-deva\grounded-segment-anything\groundingdino\groundingdino\util\inference.py", line 195, in predict_with_classes boxes, logits, phrases = predict( File "..\tracking-anything-with-deva\grounded-segment-anything\groundingdino\groundingdino\util\inference.py", line 67, in predict outputs = model(image[None], captions=[caption]) File "..\Python\Python310\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl return forward_call(*input, kwargs) File "..\tracking-anything-with-deva\grounded-segment-anything\groundingdino\groundingdino\models\GroundingDINO\groundingdino.py", line 313, in forward hs, reference, hs_enc, ref_enc, init_box_proposal = self.transformer( File "..\Python\Python310\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl return forward_call(*input, *kwargs) File "..\tracking-anything-with-deva\grounded-segment-anything\groundingdino\groundingdino\models\GroundingDINO\transformer.py", line 258, in forward memory, memory_text = self.encoder( File "..\Python\Python310\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl return forward_call(input, kwargs) File "..\tracking-anything-with-deva\grounded-segment-anything\groundingdino\groundingdino\models\GroundingDINO\transformer.py", line 576, in forward output = checkpoint.checkpoint( File "..\Python\Python310\site-packages\torch\utils\checkpoint.py", line 235, in checkpoint return CheckpointFunction.apply(function, preserve, args) File "..\Python\Python310\site-packages\torch\utils\checkpoint.py", line 96, in forward outputs = run_function(args) File "..\Python\Python310\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl return forward_call(*input, *kwargs) File "..\tracking-anything-with-deva\grounded-segment-anything\groundingdino\groundingdino\models\GroundingDINO\transformer.py", line 785, in forward src2 = self.self_attn( File "..\Python\Python310\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl return forward_call(input, **kwargs) File "..\tracking-anything-with-deva\grounded-segment-anything\groundingdino\groundingdino\models\GroundingDINO\ms_deform_attn.py", line 338, in forward output = MultiScaleDeformableAttnFunction.apply( File "..\tracking-anything-with-deva\grounded-segment-anything\groundingdino\groundingdino\models\GroundingDINO\ms_deform_attn.py", line 53, in forward output = _C.ms_deform_attn_forward( NameError: name '_C' is not defined

Could you please give me some guidance on how to solve this error?

Thank you very much in advance!

hkchengrex commented 1 year ago
  1. Try starting a new conda or venv environment. Your existing packages have conflicts.
  2. The error indicates that GroundingDINO is not properly installed, e.g., CUDA_HOME is not set.
chris-hndz commented 1 year ago

Thank you @hkchengrex again for your quick response!

I have started a new conda enviroment, this time with Python 3.8:

conda create --name=trackdeva python=3.8

and I forced to install PyTorch 1.12 with the corresponding torchvision:

pip install torch===1.12.0 torchvision===0.13.0 -f https://download.pytorch.org/whl/torch_stable.html

Then I follow the steps to install DEVA and Grounding SAM, and tried the demo:

python demo/demo_gradio.py

And when I try to test the first example video (I downloaded the video to my computer and uploaded it locally), I get an error with this message:

Traceback (most recent call last): File "...conda\envs\trackdeva\lib\site-packages\gradio\routes.py", line 508, in predict output = await route_utils.call_process_api( File "...conda\envs\trackdeva\lib\site-packages\gradio\route_utils.py", line 219, in call_process_api output = await app.get_blocks().process_api( File "...conda\envs\trackdeva\lib\site-packages\gradio\blocks.py", line 1437, in process_api result = await self.call_function( File "...conda\envs\trackdeva\lib\site-packages\gradio\blocks.py", line 1109, in call_function prediction = await anyio.to_thread.run_sync( File "...conda\envs\trackdeva\lib\site-packages\anyio\to_thread.py", line 33, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "...conda\envs\trackdeva\lib\site-packages\anyio_backends_asyncio.py", line 877, in run_sync_in_worker_thread return await future File "...conda\envs\trackdeva\lib\site-packages\anyio_backends_asyncio.py", line 807, in run result = context.run(func, args) File "...conda\envs\trackdeva\lib\site-packages\gradio\utils.py", line 641, in wrapper response = f(args, **kwargs) File "demo/demo_gradio.py", line 32, in demo_with_text devamodel, cfg, = get_model_and_config(parser) File "..\Tracking-Anything-with-DEVA\deva\inference\eval_args.py", line 65, in get_model_and_config network = DEVA(config).cuda().eval() File "...conda\envs\trackdeva\lib\site-packages\torch\nn\modules\module.py", line 689, in cuda return self._apply(lambda t: t.cuda(device)) File "...conda\envs\trackdeva\lib\site-packages\torch\nn\modules\module.py", line 579, in _apply module._apply(fn) File "...conda\envs\trackdeva\lib\site-packages\torch\nn\modules\module.py", line 579, in _apply module._apply(fn) File "...conda\envs\trackdeva\lib\site-packages\torch\nn\modules\module.py", line 602, in _apply param_applied = fn(param) File "...conda\envs\trackdeva\lib\site-packages\torch\nn\modules\module.py", line 689, in return self._apply(lambda t: t.cuda(device)) File "...conda\envs\trackdeva\lib\site-packages\torch\cuda__init__.py", line 211, in _lazy_init raise AssertionError("Torch not compiled with CUDA enabled") AssertionError: Torch not compiled with CUDA enabled

Could you please give me some guidance on how to solve this error?

Thank you very much in advance!

hkchengrex commented 1 year ago

This is a problem with your PyTorch installation. I think it is best directed to https://discuss.pytorch.org/