search

dongxingning / SHA-GCL-for-SGG

Code for paper "Stacked Hybrid-Attention and Group Collaborative Learning for Unbiased Scene Graph Generation"
MIT License
32 stars 5 forks source link

the SHA result in the paper is wrong? #2

Closed aa200647963 closed 2 years ago

aa200647963 commented 2 years ago

image This is my reproduce of TransLike-SHA, the mR@50 is 16.40, but in the paper the mR@50 is 20.8. Could you tell me the reason? Or the results of your paper are not credible?

dongxingning commented 2 years ago

Thanks for your question. Actually, SHA is not a debiasing method, so it would keep stable on the biased metric (R@k), but may be sensitive to the hardware conditions or parameter initialization on the unbiased metric (mR@k). If you still have doubts about our experimental results, we can provide the corresponding checkpoint or .log files for you to check.

rafa-cxg commented 2 years ago

Hi, I am facing performence problem too, could you please provide the corresponding .log files and checkpoints for us to check? Thanks

Thanks for your question. Actually, SHA is not a debiasing method, so it would keep stable on the biased metric (R@k), but may be sensitive to the hardware conditions or parameter initialization on the unbiased metric (mR@k). If you still have doubts about our experimental results, we can provide the corresponding checkpoint or .log files for you to check.

dongxingning commented 2 years ago

Hi, I am facing performence problem too, could you please provide the corresponding .log files and checkpoints for us to check? Thanks

We are willing to share the CHECKPOINT and LOG files, could you point out which experiment you would check (e.g., SHA-PredCls, SHAGCL-SGDet), we will provide a download link as soon as possible.