Extend this implementation API

[jurinco]

Hello Alexis

Thanks for this great Deepsort implementation.

Actually this is not about an issue, but about how to improve the use of this API.

For some problems, it is necessary to update the confidence and the class of the detection associated with the tracking at each step (in some cases they can change, mainly if different classes such as cars, trucks, buses, etc. are being handled), so when extending the class Track would be very useful if when invoking update method (which appears in several places) also passed this data (including the current frame_id):

It is currently like this:

class MultiTracker: ... def update(self, frame_id, detections, embeddings): .... track.update (next_tlbr, (mean, cov), embeddings [det_id])

Extend method API with frame_id, detection confidence and label: track.update (next_tlbr, (mean, cov), embeddings [det_id], frame_id, det.conf, det.label)

It does not have to be like that but it is the idea.

It would also be useful in the API that before the Track is removed from the lost list, a method is invoked in Track that allows for example to free up resources or perform a last minute calculation.

def _mark_lost(self, trk_id):
    self.lost[trk_id] = self.tracks[trk_id]
    if len(self.lost) > self.lost_buf_size:
        track = self.lost.popitem(last=False)
        track.lost_removed()                      # the idea is to have a chance of free up resources or take a last minute action
    del self.tracks[trk_id]

Similarly, the MOT class is the center of the whole process in the step method and it would be very useful if it called a method (by default it does nothing) do_calcs () before _draw and incrementing frame_count, in do_calc whoever extends MOT can perform operations specific to the problem.

Very good job.

Thanks

[GeekAlexis]

Thanks for your suggestions. Updating class label is not necessary because tracks and detections of different classes won't get associated. Python has automatic garbage collection so no need to clean up resources. If someone wants to add extra processing, it should be done outside mot.step() using the tracks returned by mot.visible_tracks in an application like app.py, unless they need to change the API directly.

FYI, I'm reworking the API so further suggestions are welcome.

[jurinco]

Hello Alexis,

Thanks for the reply.

What happens is that in the process I need to know the frame_count and the confidence of the last detection associated with the track, and as the API is currently, I had to create a new class that inherits from Multitracker and duplicate all the code in Multitracker.update () to include a custom call to track.update_detection (frame_id, det.conf) and thus be able to update the confidence and frame_id of the last associated detection.

This is simply to avoid having to touch the Fastmot code.

Another point where it would be good to avoid this is in the Model Yolo file, separating custom Models such as YOLOv4 in a separate directory and having Fastmot load them at startup if the directory contains files.

Thanks

[GeekAlexis]

I agree on frame count. Just curious, why do you need to know the confidence of the last associated detection? Or do you just need a detection score about the track in general?

[jurinco]

The frame_count is to determine the status of the process, for example, to know if the object has already been counted or not.

The detection confidence is as you say to know the efficiency in the detection and possibly adjust parameters.

The thing about dividing the base Yolo model and the custom ones is to avoid having to modify the Fastmot files and have greater clarity about the modifications made and avoid them being lost when synchronizing with the Fastmot versions.

[GeekAlexis]

175 restructures the API:

• Access the frame count at which a track is last updated by a detection using track.end_frame
• Visualize confidence by turning on the corresponding visualizer option in mot.json
• Inherit the base YOLO model outside FastMOT and FastMOT will be able to load it if you select it in mot.json.

For efficiency, confidence is not added in Track class.