search

zihangJiang / TokenLabeling

Pytorch implementation of "All Tokens Matter: Token Labeling for Training Better Vision Transformers"
Apache License 2.0
426 stars 36 forks source link

Token Label and ground truth #9

Closed InfinityBox closed 3 years ago

InfinityBox commented 3 years ago

The shape of 'score_map' is [2, 5, H, W], but I'm curious about why append image class label in this coordinate. https://github.com/zihangJiang/TokenLabeling/blob/5cc1461d0a07bc616f6b866313c2261dade44acc/tlt/data/dataset.py#L97

zihangJiang commented 3 years ago

Hi @InfinityBox , the class label is appended for generating the ground-truth here, https://github.com/zihangJiang/TokenLabeling/blob/5cc1461d0a07bc616f6b866313c2261dade44acc/tlt/data/mixup.py#L53-L59 and can be helpful for the case if you want to use the ground-truth label as well as the token label data for training.

lingeringlight commented 1 year ago

Hi @zihangJiang, I am also curious about the problem of dimension inconsistency. Specifically, (1) the shape of "score_map" that generated in generate_label.py is [2, 5, H, W], but the dimension of score_maps seems to be [2, H, W, 5] in "score_maps[-1,0,0,5]=target "; (2) The dimension of "label_maps_topk" in mixup.py is [batch_size, 3, H, H, 5], but I cannot find how to transform "score_maps" to "label_maps_topk", and what information is stored in the 0, 1, 2 dimension of "label_maps_topk", respectively.