Parameter Adjustment in Automatic Mode and Objects Reappearance Problem

[dhgras]

Hello, this is a great work and it has been very helpful to me! Thank you! My device has a RTX 4090 with 24GB VRAM. Currently, I am running the automatic mode demo script on multiple different RGB image sequences. There are two confusing questions for me now:

• When the frame resolution is 960 * 540 and the number of frames is below 800, I usually do not encounter CUDA out of memory error. But when dealing with larger and more image sequences (possibly 2000 4k frames), this error arises. So how should I adjust my parameters? My script's parameters are set to:

  python demo/demo_automatic.py \
  --chunk_size 1 \
  --img_path "ImgPath" \
  --detection_every 3 \
  --num_voting_frames 3 \
  --amp \
  --temporal_setting semionline \
  --size 720 \
  --output "OutputPath" \
  --suppress_small_objects  \
  --SAM_PRED_IOU_THRESHOLD 0.8 \
  --SAM_NUM_POINTS_PER_SIDE 32 \
  --SAM_NUM_POINTS_PER_BATCH 8 

• When one (or more object) is lost from the screen and reappears in subsequent frame, Deva will consider it a new object and assign it a new mask. Is there a way to solve this problem?

I hope to receive your reply or some useful suggestions from others. Thank you.

[hkchengrex]

1. OOM usually arises from having too many objects in the memory. You can reduce max_missed_detection_count to more readily delete unseen objects, reduce max_num_objects to cap the maximum number of object memory, reduce SAM_NUM_POINTS_PER_SIDE to reduce the number of detected objects, or reduce size to reduce the image resolution.
2. This is related to (1). We delete unseen objects after they have not been detected for max_missed_detection_count times. Increasing it will help re-discover some of the objects, but increase memory usage.

We are looking at ways to reduce the total memory usage but we don't have a timeline yet.

[dhgras]

Okay, I will try your suggestion. Thank you for your reply!