Roll back leads to error

[odedritov]

When I try to use the "roll back" feature, after using "Chose this mask to refine," when I try to actually click for the point prompt it invariably leads to an error.

I've been running on Colab, here's the traceback:

updating: content/Segment-and-Track-Anything/tracking_results/ChimpEmotion_Video6_Trimmed/ChimpEmotion_Video6_Trimmed_masks/00001.png (deflated 42%) choose 26 to refine choose 26 to refine final text_encoder_type: bert-base-uncased Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertModel: ['cls.predictions.transform.dense.weight', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.dense.bias', 'cls.seq_relationship.weight', 'cls.seq_relationship.bias', 'cls.predictions.bias', 'cls.predictions.transform.LayerNorm.bias']

• This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
• This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). Model loaded from ./ckpt/groundingdino_swint_ogc.pth => _IncompatibleKeys(missing_keys=[], unexpected_keys=['label_enc.weight']) SegTracker has been initialized Click choose 26 to refine Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/gradio/routes.py", line 437, in run_predict output = await app.get_blocks().process_api( File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1346, in process_api result = await self.call_function( File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1074, in call_function prediction = await anyio.to_thread.run_sync( File "/usr/local/lib/python3.10/dist-packages/anyio/to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread return await future File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 867, in run result = context.run(func, *args) File "/content/Segment-and-Track-Anything/app.py", line 251, in roll_back_sam_click predicted_mask, masked_frame = Seg_Tracker.seg_acc_click( File "/content/Segment-and-Track-Anything/SegTracker.py", line 187, in seg_acc_click refined_merged_mask = self.add_mask(interactive_mask) File "/content/Segment-and-Track-Anything/SegTracker.py", line 215, in add_mask refined_merged_mask[interactive_mask > 0] = self.curr_idx TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType'

Thank you!

[LiNO3Dy]

We changed Sam-Track to not create a new tracker during roll back. You may try pulling the latest version and see if it solves your problem.

[odedritov]

Thanks for the quick reply! Unfortunately I'm still getting the same error.

Just to make sure this should get me the latest version?

!git clone https://github.com/z-x-yang/Segment-and-Track-Anything.git
%cd /content/Segment-and-Track-Anything

[LiNO3Dy]

Hello! Thank you for raising this issue.

Based on the information you provided, it appears that clicking inside the box is necessary to select the mask that needs to be modified. As there may be multiple masks in a video, we have implemented a click-based approach, as shown in the image. In order to make Sam-Track more user-friendly, we will update the Tutorial later.

Could you please confirm if this solution resolves your issue? If you have any further questions or concerns, please do not hesitate to let us know.

[odedritov]

Thank you so much for the quick reply! I'm still getting the same issue - clicking inside the box results in an error. I did realize that this doesn't happen with the black swan demo video, but it does happen with videos I'm uploading. For example, clicking in the box triggers an error with this video:

https://github.com/z-x-yang/Segment-and-Track-Anything/assets/69921692/d5a37cab-2a1f-4acf-802e-7045d3d2c5f5

(Taken from https://commons.wikimedia.org/wiki/File:White-headed_Capuchin_(Cebus_capucinus)_video.webm)

[odedritov]

Sorry to ask again, but is there any update on when this issue might be fixed? I'm still getting the same error, and this feature would be super helpful if it worked!

[LiNO3Dy]

Thank you for your question. I ran the program using your video and did not encounter any errors. Could you please provide more detailed error information? I have recorded a video demonstration that may be helpful for you. To meet the upload size limit, I have removed some content and lowered the resolution. https://github.com/z-x-yang/Segment-and-Track-Anything/assets/97865789/bb2ce803-b9a1-484c-bdfa-b1fbdda719e8

[odedritov]

Thank you for the quick reply and the demo! This appears to be a browser-specific issue, I switched from Safari to Chrome and the feature is working now.

[joaanna]

@odedritov Could you specify which package versions are you using? I have the same issue but neither Safari nor Chrome work for me.

[odedritov]

@joaanna Unfortunately this stopped working for me again - I'm thinking this is probably because of some package update on Google Colab, but I couldn't get it to work. I've been relying on this feature for my workflow, so please let me know if you figure something out!

[joaanna]

Yes, it seems that the google colab has the same issue. @LiNO3Dy any chance you could help debug this?

[garriton]

I also encountered this problem, and later found out that it was necessary to first click a mask on the top box, then click choose this mask refine, and then adjust it on the bottom box. There was no problem