Why reid is not working at all in my case?

[RajS999]

I am trying out boxmot library and testing reid with botsort on small video clip in which a person walking on the road gets occluded behind a bill board and emerges out from other side of bill board. The whole clip is of mere 275 frames (~ 10 seconds). I tried several combinations of models / parameters, but the person always gets assigned with a new ID when he emerged our of bill board ! (I also tried botsort's official repo with yolov7 and yolox and reid, with no luck.)

I tried combination of:

• yolo8n, yolo8x,
• osnet_x0_25_msmt17.pt, clip_market1501.pt
• track buffer 30, 500

Specifically, I tried following commands:

/workspace/boxmot# python tracking/track.py  --yolo-model yolov8n.pt --reid-model osnet_x0_25_msmt17.pt --source <my-video-path> --save --conf 0.2

/workspace/boxmot# python tracking/track.py --yolo-model yolov8n --tracking-method botsort --reid-model clip_market1501.pt --source <my-video-path> --save --conf 0.2

Following with track_buffer = 500

/workspace/boxmot# python tracking/track.py --yolo-model yolov8x --tracking-method botsort --reid-model clip_market1501.pt --source <my-video-path> --save --conf 0.2

I believe I miss basic / essential understanding behind these algorithms? What am missing here? Why reid is not working at all and person always gets assigned with new id when he emerges out from other side of the bill board?

PS: fairmot (from official github repository) seem to work on this video clip out of the box when track_buffer size is increased.

[mikel-brostrom]

Once a track is marked as "lost" due to an unsuccessful association (i.e., when no detections match the track), the Kalman Filter (KF) associated with that track stops updating its state. This happens after a single unsuccessfull association.

[mikel-brostrom]

Try increasing max_age as well

[RajS999]

Am trying to make reid work with botsort. Thats why I tried botsort's official repo and thats why I specified --tracking-method botsort in second and third command. But mistakenly I did not specify it in first command, so the first command is basically invalid in the sense that it uses the default deepocsort. Sorry I did not explicitly specify this earlier ! Now I guess, botsort does not have max_age parameter, right? In that case, what might be the reason for reid to not work with it? I also tried with proximity_thresh: 0, track_buffer: 500 in botsort.yaml.

Here is my config:

appearance_thresh: 0.7636154550757781
cmc_method: sparseOptFlow
conf: 0.4828271520496597
frame_rate: 30
lambda_: 0.9764964093204093
match_thresh: 0.22182609980772128
new_track_thresh: 0.25855770885476126
proximity_thresh: 0 # 0.37262205760353817
track_buffer: 500
track_high_thresh: 0.5
track_low_thresh: 0.1

and here is my command:

python tracking/track.py --yolo-model yolov8x --tracking-method botsort --reid-model clip_market1501.pt --source <my-vid-path> --conf 0.2 --save

[RajS999]

I re-read through BoT-SORT paper and Fast-ReID paper. These are the excerpts from these papers:

• From Fast ReID paper: We propose a cross-domain method FastReIDMLT that adopts mixture label transport to learn pseudo label by multi-granularity strategy. We first train a model with a source-domain dataset and then finetune on the pre-trained model with pseudo labels of the target-domain dataset. Even after fine tuning, it achieved poorer performance for some source-target datasets for cross domain person re-identification. (Even after fine tuning the authors noted down poorer performance for some combination of source - target datasets for cross domain person re-identification as can be seen in second last line in below table.)

• From BoT-SORT paper: For the feature extractor, we trained FastReID’s SBS-50 model for MOT17 and MOT20 with their default training strategy, for 60 epochs. While we trained on the first half of each sequence and tested on the rest.

Now I have two doubts:

Q1. Does above mean we cannot simply use out of the box Re-ID models for successfully doing cross-domain person reidentification? We need to fine-tune them on our dataset before using?

Q2. What are SOTA or community preferred methods for battery powered on-device object tracking with re-identification?

[github-actions[bot]]

👋 Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs. Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!