search

z-x-yang / Segment-and-Track-Anything

An open-source project dedicated to tracking and segmenting any objects in videos, either automatically or interactively. The primary algorithms utilized include the Segment Anything Model (SAM) for key-frame segmentation and Associating Objects with Transformers (AOT) for efficient tracking and propagation purposes.
GNU Affero General Public License v3.0
2.83k stars 340 forks source link

Error in demo #93

Closed tomato18463 closed 12 months ago

tomato18463 commented 1 year ago

Hi,

When I run the colab demo, I get the following error 

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/gradio/routes.py", line 523, in run_predict
    output = await app.get_blocks().process_api(
  File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1437, in process_api
    result = await self.call_function(
  File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1109, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/usr/local/lib/python3.10/dist-packages/anyio/to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "/usr/local/lib/python3.10/dist-packages/gradio/utils.py", line 865, in wrapper
    response = f(*args, **kwargs)
  File "/content/Segment-and-Track-Anything/app.py", line 329, in tracking_objects
    return tracking_objects_in_video(Seg_Tracker, input_video, input_img_seq, fps, frame_num)
  File "/content/Segment-and-Track-Anything/seg_track_anything.py", line 100, in tracking_objects_in_video
    return img_seq_type_input_tracking(SegTracker, io_args, video_name, imgs_path, fps, frame_num)
  File "/content/Segment-and-Track-Anything/seg_track_anything.py", line 334, in img_seq_type_input_tracking
    imageio.mimsave(io_args['output_gif'], masked_pred_list, fps=fps)
  File "/usr/local/lib/python3.10/dist-packages/imageio/v2.py", line 495, in mimwrite
    return file.write(ims, is_batch=True, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/imageio/plugins/pillow.py", line 383, in write
    raise TypeError(
TypeError: The keyword `fps` is no longer supported. Use `duration`(in ms) instead, e.g. `fps=50` == `duration=20` (1000 * 1/50).

And the result won't show, I can check roll back results though. Is this some bug with the demo?

tomato18463 commented 1 year ago

Just a following up: I tried to download the example data for image-seq input, and re-uploaded it as input. I got the following error:

Archive:  /tmp/gradio/80a950b9b1caf43c83ded403b45b64d665ab27d2/840_iSXIa0hE8Ek.zip
  inflating: ./assets/840_iSXIa0hE8Ek/00000.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00001.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00002.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00003.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00004.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00005.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00006.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00007.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00008.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00009.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00010.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00011.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00012.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00013.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00014.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00015.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00016.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00017.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00018.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00019.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00020.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00021.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00022.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00023.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00024.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00025.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00026.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00027.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00028.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00029.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00030.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00031.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00032.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00033.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00034.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00035.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00036.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00037.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00038.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00039.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00040.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00041.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00042.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00043.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00044.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00045.png  
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/gradio/routes.py", line 523, in run_predict
    output = await app.get_blocks().process_api(
  File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1440, in process_api
    data = self.postprocess_data(fn_index, result["prediction"], state)
  File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1303, in postprocess_data
    self.validate_outputs(fn_index, predictions)  # type: ignore
  File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1278, in validate_outputs
    raise ValueError(
ValueError: An event handler (get_meta_from_img_seq) didn't receive enough output values (needed: 4, received: 3).
Wanted outputs:
    [image, state, image, textbox]

Is it a bug?

odedritov commented 1 year ago

Also getting the error in the first comment whenever I try to run anything; this seems to be a change in the most recent version of imageio.

Adding a block of code with this works: !pip install imageio==2.5

Run it, then do Runtime > Restart and run all, and then it should work!

EDIT: ROLLBACK STILL DOESN'T WORK

The above fix works for the initial tracking; however, when I try to use the rollback feature, I get this error:

Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/gradio/routes.py", line 442, in run_predict output = await app.get_blocks().process_api( File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1392, in process_api result = await self.call_function( File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1097, in call_function prediction = await anyio.to_thread.run_sync( File "/usr/local/lib/python3.10/dist-packages/anyio/to_thread.py", line 33, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread return await future File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 807, in run result = context.run(func, args) File "/usr/local/lib/python3.10/dist-packages/gradio/utils.py", line 703, in wrapper response = f(args, **kwargs) File "/content/Segment-and-Track-Anything/app.py", line 251, in roll_back_sam_click predicted_mask, masked_frame = Seg_Tracker.seg_acc_click( File "/content/Segment-and-Track-Anything/SegTracker.py", line 187, in seg_acc_click refined_merged_mask = self.add_mask(interactive_mask) File "/content/Segment-and-Track-Anything/SegTracker.py", line 215, in add_mask refined_merged_mask[interactive_mask > 0] = self.curr_idx TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType'

As far as I can tell this seems to be related to gradio, but downgrading gradio doesn't fix it

joaanna commented 1 year ago

Hi, I am also having this issue. I am using: gradio == '3.44.4' torch == 2.0.1+cu117

Using a different browser did not help. Could the authors provide the versions of the packages they used?

yamy-cheng commented 12 months ago

Hi,

When I run the colab demo, I get the following error

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/gradio/routes.py", line 523, in run_predict
    output = await app.get_blocks().process_api(
  File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1437, in process_api
    result = await self.call_function(
  File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1109, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/usr/local/lib/python3.10/dist-packages/anyio/to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "/usr/local/lib/python3.10/dist-packages/gradio/utils.py", line 865, in wrapper
    response = f(*args, **kwargs)
  File "/content/Segment-and-Track-Anything/app.py", line 329, in tracking_objects
    return tracking_objects_in_video(Seg_Tracker, input_video, input_img_seq, fps, frame_num)
  File "/content/Segment-and-Track-Anything/seg_track_anything.py", line 100, in tracking_objects_in_video
    return img_seq_type_input_tracking(SegTracker, io_args, video_name, imgs_path, fps, frame_num)
  File "/content/Segment-and-Track-Anything/seg_track_anything.py", line 334, in img_seq_type_input_tracking
    imageio.mimsave(io_args['output_gif'], masked_pred_list, fps=fps)
  File "/usr/local/lib/python3.10/dist-packages/imageio/v2.py", line 495, in mimwrite
    return file.write(ims, is_batch=True, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/imageio/plugins/pillow.py", line 383, in write
    raise TypeError(
TypeError: The keyword `fps` is no longer supported. Use `duration`(in ms) instead, e.g. `fps=50` == `duration=20` (1000 * 1/50).

And the result won't show, I can check roll back results though. Is this some bug with the demo?

Hi, the error is caused by the version of Pillow, you can try using "pip install Pillow==9.2.0" to resolve it.

yamy-cheng commented 12 months ago

Just a following up: I tried to download the example data for image-seq input, and re-uploaded it as input. I got the following error:

Archive:  /tmp/gradio/80a950b9b1caf43c83ded403b45b64d665ab27d2/840_iSXIa0hE8Ek.zip
  inflating: ./assets/840_iSXIa0hE8Ek/00000.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00001.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00002.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00003.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00004.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00005.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00006.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00007.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00008.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00009.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00010.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00011.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00012.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00013.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00014.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00015.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00016.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00017.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00018.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00019.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00020.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00021.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00022.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00023.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00024.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00025.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00026.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00027.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00028.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00029.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00030.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00031.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00032.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00033.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00034.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00035.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00036.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00037.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00038.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00039.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00040.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00041.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00042.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00043.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00044.png  
  inflating: ./assets/840_iSXIa0hE8Ek/00045.png  
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/gradio/routes.py", line 523, in run_predict
    output = await app.get_blocks().process_api(
  File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1440, in process_api
    data = self.postprocess_data(fn_index, result["prediction"], state)
  File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1303, in postprocess_data
    self.validate_outputs(fn_index, predictions)  # type: ignore
  File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1278, in validate_outputs
    raise ValueError(
ValueError: An event handler (get_meta_from_img_seq) didn't receive enough output values (needed: 4, received: 3).
Wanted outputs:
    [image, state, image, textbox]

Is it a bug?

Thank you for you point out, the bug has been fixed, you can update the code to use the feature.