Error when run interactive demo

[STAR-811]

Hello, thanks for your work. I installed the checkpoints and set up the environment but cannot run the demo. python -m demo.demo frames_path='/home/bbnclxy/ext16t/lyx-star/sam-pt-main/data/demo_data/bees/' query_points_path=null longest_side_length=1024 frame_stride=1 max_frames=-1 it reports error:

I would appreciate it if you could help me.

[m43]

Hej, it looks like you're encountering an error when Hydra instantiates the model. To help us get a clearer understanding of the problem, could you set the HYDRA_FULL_ERROR environment variable to 1 to display the complete stack trace, which Hydra typically hides during the model instantiation process? You can do this by running export HYDRA_FULL_ERROR=1 in your terminal anytime before executing the python command. After setting this variable, you can run your command again and share the output with us. Thanks!

[m43]

Hej @STAR-811, have you perhaps resolved your issue? Let us know if you need further guidance or if there are any updates.

[JustC0deIt]

@m43 hey,I also encountered the same problem, but after i set HYDRA_FULL_ERROR=1 in my terminal,This mistake still exists，like that

[JustC0deIt]

I have resolved the above issue because some third-party libraries from Github have not been fully installed.

[ttanad]

@m43 hey,I also encountered the same problem, but after i set HYDRA_FULL_ERROR=1 in my terminal,This mistake still exists，like that

Hi, may I ask what's your solution to the problem? I've encountered the same problem but haven't found the reason/solution yet. Thank you so much!

[m43]

Hej @ttanad, you can set the HYDRA_FULL_ERROR environment variable to 1 to see the full error. Has detectron2 been successfully installed on your system, i.e., can you run python -c 'import detectron2' without error?

[ttanad]

Hej @ttanad, you can set the HYDRA_FULL_ERROR environment variable to 1 to see the full error. Has detectron2 been successfully installed on your system, i.e., can you run python -c 'import detectron2' without error?

Thank you so much for your response! Actually, I hadn't successfully installed the detectron2 since I saw another Issue post that said it's also okay for the demo even if skipping the detectron2. And I struggled for a long time but couldn't properly install detectron2 by your instruction lines and environment. Can you kindly suggest some solution for this case?

[m43]

Here are some things to try:

1. Run export HYDRA_FULL_ERROR=1 in your console and then run the demo. Paste the complete output log here or provide a pastebin link if the output is too long.
2. Remove git+https://github.com/facebookresearch/detectron2@v0.6 from requirements.txt and re-run pip install -r requirements.txt. Try to run the demo again - will the demo work now?
3. Double check you have the minimal set of checkpoints downloaded as described here.

[ttanad]

Thank you so much for your suggestions, after properly downloading all the minimal sets, it started working and the GUI page popped out for me to choose the points. But it seems that I have some problems exiting the page with esc and seeing the result. May I ask what should I expect to see after choosing the points? Is it only the point coordinates or it's expected to see the video as a result?

(sam-pt) C:\Users\ttana\OneDrive - HKUST Connect\Data\Point_tracking\sam-pt>python -m demo.demo frames_path='${hydra:runtime.cwd}/data/demo_data/bees/' query_points_path=null longest_side_length=1024 frame_stride=1 max_frames=-1 C:\Users\ttana.conda\envs\sam-pt\lib\site-packages\hydra_internal\hydra.py:119: UserWarning: Future Hydra versions will no longer change working directory at job runtime by default.
See https://hydra.cc/docs/1.2/upgrades/1.1_to_1.2/changes_to_job_working_dir/ for more information. ret = run_job( Setting seed to 72 model: point_tracker: target: sam_pt.point_tracker.pips.PipsPointTracker checkpoint_path: ${hydra:runtime.cwd}/models/pips_ckpts/reference_model stride: 4 s: 8 initial_next_frame_visibility_threshold: 0.9 sam_predictor: sam_model: image_encoder: target: segment_anything.modeling.image_encoder.ImageEncoderViT depth: 32 embed_dim: 1280 img_size: ${ ..image_size } mlp_ratio: 4 norm_layer: partial: true target: torch.nn.LayerNorm eps: 1.0e-06 num_heads: 16 patch_size: ${ ..vit_patch_size } qkv_bias: true use_rel_pos: true global_attn_indexes:

• 7
• 15
• 23
• 31 window_size: 14 out_chans: ${ ..prompt_embed_dim } prompt_encoder: target: segment_anything.modeling.prompt_encoder.PromptEncoder embed_dim: ${ ..prompt_embed_dim } image_embedding_size:
• ${ ...image_embedding_size }
• ${ ...image_embedding_size } input_image_size:
• ${ ...image_size }
• ${ ...image_size } mask_in_chans: 16 mask_decoder: target: segment_anything.modeling.mask_decoder.MaskDecoder num_multimask_outputs: 3 transformer: target: segment_anything.modeling.transformer.TwoWayTransformer depth: 2 embedding_dim: ${ ...prompt_embed_dim } mlp_dim: 2048 num_heads: 8 transformer_dim: ${ ..prompt_embed_dim } iou_head_depth: 3 iou_head_hidden_dim: 256 target: sam_pt.modeling.sam.SamHydra checkpoint: ${hydra:runtime.cwd}/models/sam_ckpts/sam_vit_h_4b8939.pth prompt_embed_dim: 256 image_size: 1024 vit_patch_size: 16 image_embedding_size: 64 pixel_mean:
  • 123.675
  • 116.28
  • 103.53 pixel_std:
  • 58.395
  • 57.12
  • 57.375 target: segment_anything.predictor.SamPredictor target: sam_pt.modeling.sam_pt.SamPt sam_iou_threshold: 0.7 iterative_refinement_iterations: 12 positive_point_selection_method: kmedoids negative_point_selection_method: mixed positive_points_per_mask: -1 negative_points_per_mask: -1 add_other_objects_positive_points_as_negative_points: true point_tracker_mask_batch_size: 5 use_patch_matching_filtering: false patch_size: 3 patch_similarity_threshold: 0.01 use_point_reinit: false reinit_point_tracker_horizon: 8 reinit_horizon: 8 reinit_variant: reinit-on-horizon-and-sync-masks logging: wandb: entity: null project: demo tensorboard: true log_code_path: ${hydra:runtime.cwd}/sam_pt debug: false exp_id: debug_72_2023.09.27_00.06.50 frames_path: ${hydra:runtime.cwd}/data/demo_data/bees/ query_points_path: null longest_side_length: 1024 frame_stride: 1 max_frames: -1 seed: 72 annot_size: 16 annot_line_width: 6

wandb: Currently logged in as: ttanad664. Use wandb login --relogin to force relogin wandb: wandb version 0.15.11 is available! To upgrade, please run: wandb: $ pip install wandb --upgrade wandb: Tracking run with wandb version 0.15.3 wandb: Run data is saved locally in C:\Users\ttana\OneDrive - HKUST Connect\Data\Point_tracking\sam-pt\outputs\2023-09-27\00-06-50\wandb\run-20230927_000652-rxqaxdhh wandb: Run wandb offline to turn off syncing. wandb: Syncing run debug_72_2023.09.27_00.06.50 wandb: View project at https://wandb.ai/ttanad664/demo wandb: View run at https://wandb.ai/ttanad664/demo/runs/rxqaxdhh C:\Users\ttana.conda\envs\sam-pt\lib\site-packages\torch\nn\functional.py:3499: UserWarning: The default behavior for interpolate/upsample with float scale_factor changed in 1.6.0 to align with other frameworks/libraries, and now uses scale_factor directly, instead of relying on the computed output size. If you wish to restore the old behavior, please set recompute_scale_factor=True. See the documentation of nn.Upsample for details. warnings.warn( How many positive points? 2 How many negative points? 1 Added positive point 1 at (272, 215) Added positive point 2 at (224, 201) Added negative point 3 at (401, 236) Added query points for mask 1, you can now select points for the next mask or press esc to exit.

Added positive point 1 at (274, 174) Added positive point 2 at (251, 163) Added negative point 3 at (361, 271) Added query points for mask 2, you can now select points for the next mask or press esc to exit.

Added positive point 1 at (907, 348) Added positive point 2 at (911, 326) Added negative point 3 at (802, 340) Added query points for mask 3, you can now select points for the next mask or press esc to exit.

Added positive point 1 at (938, 332) Added positive point 2 at (891, 339) Added negative point 3 at (767, 334) Added query points for mask 4, you can now select points for the next mask or press esc to exit.

Added positive point 1 at (904, 277) Added positive point 2 at (984, 286) Added negative point 3 at (917, 115) Added query points for mask 5, you can now select points for the next mask or press esc to exit.

Traceback (most recent call last): File "C:\Users\ttana.conda\envs\sam-pt\lib\runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None, File "C:\Users\ttana.conda\envs\sam-pt\lib\runpy.py", line 87, in _run_code exec(code, run_globals) File "C:\Users\ttana\OneDrive - HKUST Connect\Data\Point_tracking\sam-pt\demo\demo.py", line 372, in main() File "C:\Users\ttana.conda\envs\sam-pt\lib\site-packages\hydra\main.py", line 94, in decorated_main _run_hydra( File "C:\Users\ttana.conda\envs\sam-pt\lib\site-packages\hydra_internal\utils.py", line 394, in _run_hydra _run_app( File "C:\Users\ttana.conda\envs\sam-pt\lib\site-packages\hydra_internal\utils.py", line 457, in _run_app run_and_report( File "C:\Users\ttana.conda\envs\sam-pt\lib\site-packages\hydra_internal\utils.py", line 220, in run_and_report return func() File "C:\Users\ttana.conda\envs\sam-pt\lib\site-packages\hydra_internal\utils.py", line 458, in lambda: hydra.run( File "C:\Users\ttana.conda\envs\sam-pt\lib\site-packages\hydra_internal\hydra.py", line 119, in run ret = run_job( File "C:\Users\ttana.conda\envs\sam-pt\lib\site-packages\hydra\core\utils.py", line 186, in run_job ret.return_value = task_function(task_cfg) File "C:\Users\ttana\OneDrive - HKUST Connect\Data\Point_tracking\sam-pt\demo\demo.py", line 45, in main rgbs, positive_points_per_mask, query_points = load_data(cfg) File "C:\Users\ttana\OneDrive - HKUST Connect\Data\Point_tracking\sam-pt\demo\demo.py", line 96, in load_data return data_loader( File "C:\Users\ttana\OneDrive - HKUST Connect\Data\Point_tracking\sam-pt\demo\demo.py", line 341, in load_demo_data_interactive k = cv2.waitKey(1) KeyboardInterrupt wandb: Waiting for W&B process to finish... (failed 255). Press Ctrl-C to abort syncing. wandb: wandb: Run summary: wandb: work_dir C:\Users\ttana\OneDr... wandb: wandb: View run debug_72_2023.09.27_00.06.50 at: https://wandb.ai/ttanad664/demo/runs/rxqaxdhh wandb: Synced 6 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) wandb: Find logs at: .\wandb\run-20230927_000652-rxqaxdhh\logs

[m43]

After you press escape, you should get a message saying Below is the query point data... and then the inference would run, followed by displaying the segmentation result in the same cv2 window. Since cv2 does not recognize you clicking escape, make sure you have the cv2 window in focus when clicking escape (e.g., clicking on the titlebar puts a window into focus). If this does not work, I'd recommend you consider using a different termination criteria, here is the entry point.