JialianW / TraDeS

Track to Detect and Segment: An Online Multi-Object Tracker (CVPR 2021)
MIT License
553 stars 108 forks source link

Label #31

Closed Leo63963 closed 3 years ago

Leo63963 commented 3 years ago

Hi Could you please tell me the reason form in your code below: https://github.com/JialianW/TraDeS/blob/3eafd249ca0f18af8000d5798d4c552a0bd627ec/src/lib/dataset/generic_dataset.py#L679 https://github.com/JialianW/TraDeS/blob/3eafd249ca0f18af8000d5798d4c552a0bd627ec/src/lib/dataset/generic_dataset.py#L680 which are used for making the labels of cost volume. It seems that in the code: https://github.com/JialianW/TraDeS/blob/3eafd249ca0f18af8000d5798d4c552a0bd627ec/src/lib/dataset/generic_dataset.py#L677 https://github.com/JialianW/TraDeS/blob/3eafd249ca0f18af8000d5798d4c552a0bd627ec/src/lib/dataset/generic_dataset.py#L678 the labels for h-hm and w-hm as already made.

JialianW commented 3 years ago

You need to go to the loss computation process to have a look. The center, which is positive sample, has a different label from the negative samples. Same happens in the cls heatmap loss computation in original CenterNet.

Leo63963 commented 3 years ago

Thanks for the reply. Yes, it is. However, in my opinion the labels in CenterNet and in TraDeS are different in that, the labels _ret['h_hmdown'] and _ret['w_hmdown'] are used for the proposed cost volume, however, the labels in CenterNet are Gaussian heatmaps. Thus, could you please give me details ? Especially for the https://github.com/JialianW/TraDeS/blob/3eafd249ca0f18af8000d5798d4c552a0bd627ec/src/lib/dataset/generic_dataset.py#L679 and https://github.com/JialianW/TraDeS/blob/3eafd249ca0f18af8000d5798d4c552a0bd627ec/src/lib/dataset/generic_dataset.py#L680 Why they are 0 here?
In the paper, the Y_xykl in the paper is 1 if the object (x, y) at frame t located at (k, l) in the previous frame, otherwise 0. Thus, I wonder if 0 here means the negative samples? Thanks.

JialianW commented 3 years ago

No, 0 here doesn't mean negatives. This is why I recommend you go to loss to look. out * (1-target), if target is 0, it will attend the softmax. If it is 1, it won't attend. This process is related to your last question. Pls look how the loss is calculated. In principle, all targets should be 0. But we want nearby pixels around the center to less attend softmax, as they are ambiguous. So we take advantage of the gaussian heatmap to determine how less it will attend. And for those negatives who are beyond the gaussian kernel radius, all their labels are 1.

Leo63963 commented 3 years ago

Thank you so much for your reply.