Algortihm pipeline

[circularProteus]

Hi, I'm trying to understand the overall behavior of this project... It works but I don't know how. From what I understood:

• Using Yolo you retrive the bb coordinates for person
• Invoking SORT to "concatenate" information between frames
• display correct bb with ID (is it correct??)

what happen to this pipeline (assuming it's correct) when deep_SORT is set to True?

[obendidi]

It's quite simple, this project is just a merge between two different projects :

• Darkflow : for object detection (you can use any other object detector for that )
• Sort/Deep_Sort : for tracking of the previously detected objects

the process is also simple :

1. for each frame , get detection from YOLO
2. for each frame, feed the detection to sort so that it compares them to previous detection using kalman filters mainly , to finally get the tracking Id for each detection
3. Repeat

Deep_sort is based on sort , plus a deep encoder module that encodes features of each detected person to compare them between the frames , helps to keep the tracking going when there is object occlusion in some frames in between

[circularProteus]

thanks for your answer... I understand the detection pipeline, another question... according to the section "Using darkflow from another python application" (see darkflow repository) I know, using tfnet.return_predict(image) I obtain a Json output containing bb informations. In order to embed sort/dark_sort what should I do?

[obendidi]

you just take that bb info , format it into a list of boxes and feed it to deep sort or sort

[circularProteus]

ok thanks, I'm now able to use SORT (by using track_bbs_ids = mot_tracker.update(detections)). The question is how to use the same "detection" with deep_SORT. By now I'm able to track items "off-line" (generate det.txt + img.jpg, feed into generate_detection.py to obtain npy file and finally use this for the visualization).. I'd like to perform this in on-line mode

[mandeepubc]

@alessi89 Did you figure out how to use deep_SORT?