ID sequence of ground truth and prediction

[aia39]

Hello, I was successfully able to execute your code but still, concern about one thing is that do we need to pass the same ID sequence in prediction txt file as ground truth txt file? For instance, '0' ID in prediction has bounding box of '1' ID in ground truth and same also for '1' ID in prediction has bounding box of '0' ID in ground truth.

Another query is that does ID number of prediction txt file has to be same as ground truth txt file? Or this script works regardless of ID number, it just cares if an ID number is tracked or not is txt files? For instance, '0' ID in ground truth has the same bounding box in '37' ID in prediction txt file but it is tracking '37' nicely in all files. Is this okay for correct result?

[cheind]

Hey! I am not sure I can follow your question. First, recognize that any file format supported in motmetrics is converted to calls to an accumulator that processes the information stored in files. Thus, it might be easier for your application to first investigate what the accumulator expects to perform tracking evaluation. This is documented here

https://github.com/cheind/py-motmetrics#usage

For every frame it assumes three things

• a set of unique names (ids) for ground truth objects
• a set of unique names (ids) that your tracker outputs
• a distance matrix that computes pairwise distances between ground truth objects and tracker outputs.

The last bullet point is usually computed via one of the following helper functions https://github.com/cheind/py-motmetrics#computing-distances

Next, you will find that the application you are using performs similar steps for all frames found in the given ground-truth and tracker files. https://github.com/cheind/py-motmetrics/blob/develop/motmetrics/apps/eval_motchallenge.py

The way your files need to be setup depends on the dataset you are evaluating for. I am assuming the default, which is parsed here https://github.com/cheind/py-motmetrics/blob/d261d16cca263125b135571231011ccf9efd082b/motmetrics/io.py#L48 and expects the following columns to be present in your files 'FrameId', 'Id', 'X', 'Y', 'Width', 'Height', 'Confidence', 'ClassId', 'Visibility', 'unused'

Hope this helps

[ariqhadi]

Hi, I am having trouble with knowing or giving the ID of the object in my ground truth data, because I want to use the tracker to track sperm movement, thus making it so hard to manually giving ID to each sperm object because of the similarity of the object.

Do you have any suggestion regarding the tools that I can use to give my ground truth object an ID, or can I just give random ID for every object in my ground truth data (eg. 1,2,3 for every object in every frame).

Thank you

[aia39]

@ariqhadi Hello. I can share my work how I solve this, maybe it can be helpful for you. I am using a detector to get bounding box of each object and I also have bounding box as ground truth as you see above I have provided an image. So I ran a tracker algorithm on both detections and ground truth bounding box which generates ID for me. I think running a proper tracker algorithm may do the work for you.

[cheind]

Could you clarify the following: is your problem a) that you, as a human, cannot reliably assign a unique ID to visual sperm data or b) you don't know how to come up with good names to label your data?

b) could easily solved using random/incremental IDs a) takes more thought.

[aia39]

Hello, @cheind Thank you for your previous reply. I have one more question regarding the task of computing distance. Here how can I determine "max_d2" parameter from "C = mm.distances.norm2squared_matrix(o, h, max_d2=5.) " from following link you provided ?

• a distance matrix that computes pairwise distances between ground truth objects and tracker outputs.

The last bullet point is usually computed via one of the following helper functions https://github.com/cheind/py-motmetrics#computing-distances

[ariqhadi]

Thank you @cheind and @aia39, I tried both your advice by creating the ID with incremental IDs and using tracker for the groundtruth box to get the IDs. both seems work but the final results is very small like in the image below.

what do you think is the problem here?

thanks

[pandey-anurag]

@cheind Reiterating @aia39's issue here. The sequence of ground truth track ID might be different from tracker's track ID. For instance, GT denotes two object on left and right as track ID 0 and 1 respectively, however, my tracker might assign them the ID as 1 and 0 respectively and continue to do so for frames following. Although the IoU can indicate the association between the track IDs, however, they can still be classified under Identity Switches. Can ypu provide some clarification on this?

[jvlmdr]

Hey. The ground-truth IDs and predicted IDs in the input do not need to match. The toolkit figures out the matching between tracks (IDF1 uses a single, per-video matching of entire tracks whereas MOTA permits the matching to change over time and counts the number of changes). The IDs in the input are only used to group boxes into tracks.

To make this clear, you could apply a re-mapping to the predicted IDs and it would have no effect on the results (assuming an injective mapping).

[jvlmdr]

@pandey-anurag If the two ground-truth tracks had IDs 0 and 1 while the two predicted tracks had IDs 1 and 0 throughout the sequence, there should be no identity switches. Would you like to share the inputs?

[pandey-anurag]

@jvlmdr Thanks for your input. For the identity switch, yes, consistent IDs should not contribute to ID switch, however, if for some reason tracker does not produce consistent IDs, it might contribute to ID switches.

[jvlmdr]

OK, I will close this issue for now as I think we have addressed the question. Feel free to re-open with a comment if this is not the case. @pandey-anurag @ariqhadi @aia39