How to generate DHN_data on the custom data?

[ghost]

I want to apply your great work to my private data.
I have run your code on MOT successfully. But I met some difficulties when generating DHN_data on my private data.
I have read your paper but am still confused. Could you explain how to generate the DHN_data on MOT in detail and provide a README for the DHN_data? Could you please share the code for generating the DHN_data? Thanks! @yihongXU

[yihongXU]

Hi, Thank you for your interest! DHN data is easy to be generated. I was taking gt bboxes and detections provided by MOT15,16 and MOT17, and I calculated the distances (same as dist during training,https://gitlab.inria.fr/yixu/deepmot/-/blob/master/train_tracktor/src/tracktor/tracker.py line 392) between them -> a distance matrix is made. I augumented by thresholding them with different values. As for ground-truth labels, we input these distances into Hungarian algorithm, and get the ground-truth binary assignment for each distance.

You can surely make your custom data, and define your own distance.

Hope that can help.

[ghost]

@yihongXU Hi. Thanks for your reply! I am not sure what your words "augument by thresholding them with different values" mean. Does it mean that zero out the value below the threshold and keep the value above the threshold? e.g. The distance matrix is [0.3, 0.5, 0.2], the stochastic threshold is [0.6, 0.4, 0.1], then the distance matrix is augmented to [0.0, 0.5, 0.2]. Is that right?

[yihongXU]

Hi,

Suppose that your distance matrix values are [0,1], thresholding means that you threshold out those large distance that cannot be considered as a match. In your example, you threshold the distance matrix [0.3, 0.5, 0.2] by a value [0,1], let say 0.4, then it will become [0.3, inf, 0.2]. Inf referring to a big value ( like in https://github.com/cheind/py-motmetrics, you set it to np.nan) in order not to assign when you input it to Hungarian Algorithm.

[ghost]

Thanks! I got it. The output shape of https://gitlab.inria.fr/yixu/deepmot/-/blob/master/train_tracktor/src/tracktor/tracker.py is [N, M], where N is the number of the objects and M is the number of the detections on this frame. But the shape DHN_data/*/*.npy is all (10, 9, 15). What do the three dimensions “10, 9, 15” represent separately? And the names of subdirectories are all in the format of “x_y”. What do x, y represent here?

[yihongXU]

Hi,

10 is the batch size, I grouped the matrices having same size to a batch, just to accelerate a bit the training, otherwise you need to train it one by one since the input size varies.

x_y represents h,w of the matrix, as I said I regrouped the matrices by their sizes.

[ghost]

Thanks! I got it. But I am still a little confused. There are train_DHN/DHN_data/train/1_6/2DMOT2015_1_6_7_m.npy and rain_DHN/DHN_data/train/1_6/2DMOT2015_1_6_12_m.npy. What does that number (7 and 12 here) represent?

[yihongXU]

Hi,

7 and 12 are just indexes for numbering the matrices. The reason why they are not continuous is that I picked randomly some of the matrices to the validation set.

[972821054]

@magamig @yihongXU Hello, have you successfully made the data for DHN? I would appreciate if you could inform me about the process of making DHN!