search

MILVLG / bottom-up-attention.pytorch

A PyTorch reimplementation of bottom-up-attention models
Apache License 2.0
294 stars 76 forks source link

Saving of extracted features #86

Closed RomeroBarata closed 2 years ago

RomeroBarata commented 2 years ago

Hi,

Thanks for your excellent work. During the feature extraction and saving process, I was wondering why this line here https://github.com/MILVLG/bottom-up-attention.pytorch/blob/a8ad05eaef0e75956df056588d789d595d2c4a24/utils/extract_utils.py#L70 is range(1, scores.shape[1]) instead of range(scores.shape[1]). The background label has already been removed from scores so that line would be ignoring the first class from VG?

Kind regards, Romero

JimLee4530 commented 2 years ago

Hi~ I remember that scores is taken directly from the output of roi_head during the feature extraction, there are 1601 dimensions, and the background label has not removed.

https://github.com/MILVLG/bottom-up-attention.pytorch/blob/a8ad05eaef0e75956df056588d789d595d2c4a24/bua/caffe/modeling/roi_heads.py#L176

https://github.com/MILVLG/bottom-up-attention.pytorch/blob/a8ad05eaef0e75956df056588d789d595d2c4a24/bua/caffe/modeling/fast_rcnn.py#L192-L200

RomeroBarata commented 2 years ago

I see, I think the mismatch is from the caffe/d2 modes.

In the d2 version the background is removed https://github.com/MILVLG/bottom-up-attention.pytorch/blob/a8ad05eaef0e75956df056588d789d595d2c4a24/bua/d2/modeling/roi_heads.py#L255-L264

MIL-VLG commented 2 years ago

fixed the d2 feature extraction problem in #88