multi-class and multi-object tracking?

[albertchristianto]

hi @nwojke, Is it possible to modify this code for multi-class and multi-object tracking purposes? If it is possible, can you give me a suggestion about which parts of the code that should be modified? thank you very much. best regards, Albert Christianto

[faisalshahbaz]

Hi @albertchristianto I want to do something similar. I was stuck in a error. if you want, i can share my repo with you. we could work together. Connect with me on linkedin. https://www.linkedin.com/in/faisalshahbaz/

[albertchristianto]

Hi @faisalshahbaz, Actually I have modified this repo and made it for multi-class and multi-object tracker. best regards, Albert Christianto

[faisalshahbaz]

Hi @albertchristianto Thanks for quick response, If its possible, please share with me your repo.

[albertchristianto]

@faisalshahbaz, i am sorry i don't have repo for this topic because i change yolov3 from alexeyAB. it is a complicated pipeline and i am kind of busy to upload and documented the pipeline. But i can give you my suggestion about how i do it. you can change the min_cost_matching function from linear_assignment.py with this function +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ def min_cost_matching( distance_metric, max_distance, tracks, detections, track_indices=None, detection_indices=None): """Solve linear assignment problem.

Parameters
----------
distance_metric : Callable[List[Track], List[Detection], List[int], List[int]) -> ndarray
    The distance metric is given a list of tracks and detections as well as
    a list of N track indices and M detection indices. The metric should
    return the NxM dimensional cost matrix, where element (i, j) is the
    association cost between the i-th track in the given track indices and
    the j-th detection in the given detection_indices.
max_distance : float
    Gating threshold. Associations with cost larger than this value are
    disregarded.
tracks : List[track.Track]
    A list of predicted tracks at the current time step.
detections : List[detection.Detection]
    A list of detections at the current time step.
track_indices : List[int]
    List of track indices that maps rows in `cost_matrix` to tracks in
    `tracks` (see description above).
detection_indices : List[int]
    List of detection indices that maps columns in `cost_matrix` to
    detections in `detections` (see description above).

Returns
-------
(List[(int, int)], List[int], List[int])
    Returns a tuple with the following three entries:
    * A list of matched track and detection indices.
    * A list of unmatched track indices.
    * A list of unmatched detection indices.

"""
if track_indices is None:
    track_indices = np.arange(len(tracks))
if detection_indices is None:
    detection_indices = np.arange(len(detections))

if len(detection_indices) == 0 or len(track_indices) == 0:
    return [], track_indices, detection_indices  # Nothing to match.

cost_matrix = distance_metric(
    tracks, detections, track_indices, detection_indices)
cost_matrix[cost_matrix > max_distance] = max_distance + 1e-5

row_indices, col_indices = linear_assignment(cost_matrix)

matches, unmatched_tracks, unmatched_detections = [], [], []
for col, detection_idx in enumerate(detection_indices):
    if col not in col_indices:
        unmatched_detections.append(detection_idx)
for row, track_idx in enumerate(track_indices):
    if row not in row_indices:
        unmatched_tracks.append(track_idx)

for row, col in zip(row_indices, col_indices):
    track_idx = track_indices[row]
    detection_idx = detection_indices[col]
    if cost_matrix[row, col] > max_distance:
        unmatched_tracks.append(track_idx)
        unmatched_detections.append(detection_idx)

#######################this is my change elif detections[detection_idx].cls_id!=tracks[track_idx].cls_id:

delete the matched detection if the detetcion class isn't matched

        unmatched_tracks.append(track_idx)
        unmatched_detections.append(detection_idx)

#################################################### else: matches.append((track_idx, detection_idx)) return matches, unmatched_tracks, unmatched_detections +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

[faisalshahbaz]

Thank you @albertchristianto , so nice of you. I am trying, I have an error. https://drive.google.com/file/d/1-oo3fOybCFEgXsxxAGR90VpMzFNvmtMy/view?usp=sharing

[albertchristianto]

you should add the class id information in the tracker, track, and detection class. then you are good to go

[tiberium24]

Did you retrain the re-identification for some other class? How can you use the trained on pedestrians to distinguish other classes such as different cars?

[BoHuang-ecr]

Thank you @albertchristianto , so nice of you. I am trying, I have an error. https://drive.google.com/file/d/1-oo3fOybCFEgXsxxAGR90VpMzFNvmtMy/view?usp=sharing

Did you got it working at end? Multi class + Multi object tracking?

[bansilol]

Hi Guys! I am trying to convert the code to a multi class tracker, If someone was able to do it, it would be a great help if you share ideas or code snippets. Thanks!