Filter out background in bboxes by segmentation before classification

[mikel-brostrom]

Hi @KaiyangZhou. I was wondering if you ever did any experiments regarding the possibility of filtering out the background in the people bounding boxes by using instance segmentation models and how that changed the ReID performance. Some of the bounding boxes could contain people in the background or parts of them which could confuse the model, leading to less accurate identification results. I have some results here for tracking and some example images of what I input to your ReID models: https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/Masked-detection-crops-vs-regular-detection-crops-for-ReID-feature-extraction. IDF1 (the ratio of correctly identified detections over the average number of ground-truth and computed detections) increases quite a lot given that the models have not been trained this way. Do you think that retraining the models with greyed-out background would increase the classification performance dramatically?

[dheerajluffy]

~hi mikel , I am having hard time using torchreID for the obvious purpose, it is not re assigning the same id for a person if he/she re enters to the frame, I guess the segmented images make it more robust too, any idea on how to go about it?

[mikel-brostrom]

My experiment results on this can be seen here: https://github.com/mikel-brostrom/yolo_tracking/wiki/Masked-detection-crops-vs-regular-detection-crops-for-ReID-feature-extraction The boost on IDF1 is there but is not enormous. I would guess that if you already have reid issues, segmentating the background won't be enough.

[dheerajluffy]

so do you got any good results from using the torchreID? I am tying to use different repos to achieve atleast low noise in assigning the ids, even tried to increase the max_age to 30000 but still no luck