Closed JingweiJ closed 4 years ago
Thanks for your interest in our work. Firstly, make sure the newest version, because we merged the human and object branch last Sunday. Secondly, for 'No param', we are sorry that the missing update for opt.py file, but it does not cause the accuracy loss. Looking forward to your Great action genome dataset!
Thanks for your reply! I double-checked that i've been using the latest codebase. I also evaluated on the model_best.pth
in dla34, where i got similar mAPs, but way lower than expected.
Looking at the stored output best_predictions.json
, i found that there are many noisy boxes and many of them are heavily overlapped. Is it because that there's no nms and no thresholding low-confident detections?
The mAPs evaluated by our provided scripts are a bit lower than (about 0.5% mAP) the official evaluation script.
Using nms can definitely improve the performance, but it will cause a slow inference-speed. It is proved by [Objects as points] (https://arxiv.org/abs/1904.07850) that it will not produce many bounding-boxes with highly overlapping. The highly overlapping bounding-boxes may be caused by the HICO-Det annotations, where the same object/human will be annotated several times.
Got it. Thanks for the explanation and reference. My main concern is the discrepancy between the results i got (3.9 mAP) compared with the expected numbers. Still figuring out what is missing.
I have checked the code and reproduced based on the released code from scratch just now. And, I reproduce the performance and achieved the mAPs reported by us (19.94). I wonder more details.
Some details of my env: python 2.7, pytorch 0.5.0, cuda 9.0, cudnn 7.3.1
might be due to the pytorch version? i'll build a pytorch 0.4.1 and try again
I got. It main caused by the python version, where i used python 3.6, pytorch 0.4.1. I am sorry that we forget to give the right python version.
ah i see. Since there are a few from __future__ import
, i thought i should use python 2. I'll try and update
python 3.6 works! Thanks! Closing the issue
Hi Yue,
Thanks for you nice work! I tried to run the evaluation codes on your shared model (
model_last.pth
of dla34 in https://drive.google.com/drive/folders/1K0H05nSUOCq939tmvBRJjskdPSLy1U-U, I renamed itmodel_last.dla34.pth
), however i failed to reproduce the numbers. Below are the command i run and stdout/stderr i got (file path masked). Do you have some thoughts on what might be wrong? There seems to be some parameters missing in the.pth
file, is it desired?