hkchengrex / Cutie

[CVPR 2024 Highlight] Putting the Object Back Into Video Object Segmentation
https://hkchengrex.com/Cutie/
MIT License
598 stars 61 forks source link

GUI and scripting demo get different tracking results with a same initial mask #85

Closed HitLyn closed 1 week ago

HitLyn commented 1 week ago

I get different results when using the interactive GUI and running the script with the same binary mask. It looks like some threshold configurations are different between the two methods. How to get the same tracking result as the GUI, it is cleaner and more stable.

For reference, I use the upload mask function in the interactive GUI, and the mask image is saved from the same binary array used in the script, but the results are different. Are there any specific configurations used in the GUI that are different from the given script?

Thanks for the great project.

https://github.com/hkchengrex/Cutie/assets/25073978/74ff8c5e-aa40-41c9-8b8c-a701ff19b7db

https://github.com/hkchengrex/Cutie/assets/25073978/1e0324d1-e2a6-45e7-b851-db18c7f86834

hkchengrex commented 1 week ago

One reason might be that we resize the frames such that the shorter side has 480 pixels in the GUI but we do not do the same for the scripting demo. Can you verify if that's the case? Please try adding

processor.max_internal_size = 480

right after the initialization of InferenceCore

HitLyn commented 1 week ago

One reason might be that we resize the frames such that the shorter side has 480 pixels in the GUI but we do not do the same for the scripting demo. Can you verify if that's the case? Please try adding

processor.max_internal_size = 480

right after the initialization of InferenceCore

This works! Thanks a lot. The _align_corners_ parameter here needs to be dropped to avoid the error: _valueerror: aligncorners option can only be set with the interpolating modes: linear | bilinear | bicubic | trilinear https://github.com/hkchengrex/Cutie/blob/a9cad9acdd3bcb027e48e276547b9766312d8326/cutie/inference/inference_core.py#L224

hkchengrex commented 1 week ago

Thanks for the report! I have fixed this bug and added the option to change the max_internal_size in the scripting demo.