search

yrcong / STTran

Spatial-Temporal Transformer for Dynamic Scene Graph Generation, ICCV2021
MIT License
181 stars 34 forks source link

clean_class、score threshold、evaluation metrics (with, no, semi) #55

Closed lianglili closed 8 months ago

lianglili commented 10 months ago

Hello, thank you very much for your work! I have the following inquiries regarding your code:

In the Sttran.py file, the method clean_class(self, entry, b, class_idx) defines class_idx = 5, 8, 17. Could you please explain the reasoning behind these specific values?

In the object_detector.py file, the code snippet "inds = torch.nonzero(scores[:, j] >= 0.1).view(-1)" sets the threshold to 0.1. When using your provided pretrained FasterRCNN model for Action Genome, this results in an empty detection output (inds). Could you shed some light on this issue?

I adjusted the threshold to 0.05, and the model started working correctly. However, unfortunately, the evaluation metrics (with, no, semi) all yield a result of 0.0. Could you kindly address this matter?

I eagerly await your response and once again express my gratitude for your exceptional work. Thank you!

yrcong commented 8 months ago

There are some problems with the object categories of the Action Genome. These specific categories overlap with others (e.g. notebook <-> paper). We deactivate these indices since they cause troubles in the evaluation.

"inds = torch.nonzero(scores[:, j] >= 0.1).view(-1)" I am not so sure about this. It sounds like a bug. The code should be related to NMS. 0.1 is a hyperparameter after several tests.