search

KaihuaTang / Scene-Graph-Benchmark.pytorch

A new codebase for popular Scene Graph Generation methods (2020). Visualization & Scene Graph Extraction on custom images/datasets are provided. It's also a PyTorch implementation of paper “Unbiased Scene Graph Generation from Biased Training CVPR 2020”
MIT License
1.07k stars 229 forks source link

Question about resampling #23

Closed Scarecrow0 closed 4 years ago

Scarecrow0 commented 4 years ago

Hi!

Thanks for your interesting work and this nice codebase! 6666

I have some questions about the details of the resampling baseline. In the paper, I saw that the resampling is meant "Rare categories were up-sampled by the inversed sample fraction during training." The scene graph generation is a detection problem, you can upsample the data by repeating the images or repeat the relationship in one batch. If upsampled by repeating images, one image may contain multiple relationships, which means the common categories may be upsampled at the same time. If upsampled by repeating relationships means to scale up the loss of the rare relationship in each batch by multiply some number.

The code and the reference [3] in the paper seem to have not shown this resampling process on the scene graph generation task specifically.

Could you tell the detail about this resampling technic?

Looking forward to your reply.

KaihuaTang commented 4 years ago

In my implementation, I considered an entire image as a whole. The probability to up-sample an image is based on the average probability to up-sample all ground-truth relationships in this image.

This implementation is not elegant and I don't want to encourage later researchers to follow so I'm ashamed of putting this up-sampling code inside the codebase.

However, one of my collaborator Xinting also published a paper in CVPR 2020 called "Learning to Segment the Tail" Paper Link [https://arxiv.org/abs/2004.00900], Code Link [https://github.com/JoyHuYY1412/LST_LVIS]. She proposed a more elegant resampling strategy for detection/segmentation task.

image

Scarecrow0 commented 4 years ago

Looks good, I'm going to check it out.

Thanks for your rapid reply!