search

DirtyHarryLYL / HOI-Learning-List

A list of Human-Object Interaction Learning.
586 stars 57 forks source link

Update benchmark results of zero-shot settings #3

Closed yeliudev closed 3 years ago

yeliudev commented 3 years ago

Hi @DirtyHarryLYL, thanks for maintaining this useful repo! We would like to update the re-implemented results of the model 'ConsNet', and incorporate the benchmark results of zero-shot HOI detection. We also released a general Python API for the HICO-DET dataset, which contains several useful functions for data loading & processing, human-object pair IoU & NMS calculation, and standard evaluation. Hope it can provide some help for the whole HOI community.

DirtyHarryLYL commented 3 years ago

Hi, I have merged the commit. I find the performance given GT boxes is very impressive, is this score reported in the paper or the new ConsNet repo? Thanks!

yeliudev commented 3 years ago

Thanks for your reply! Yes we've updated our paper and the new version will be available on arXiv soon. But we did not report the GT boxes results in the paper nor the repo. The results reported in your repo are obtained using this config, this detector and this checkpoint, which are exactly the same as the 4th model in the model zoo. Minor changes in the tools/build_dataset.py are also needed to purge non-GT boxes. I'll clean and push the code soon (maybe tonight or tomorrow) since you are interested.

yeliudev commented 3 years ago

I think ConsNet's sharp performance drop from GT to COCO detectors comes from the hard negative samples. We've already incorporated your TIN design to suppress these samples. It helps but the remaining neg samples are still hard to classify.

DirtyHarryLYL commented 3 years ago

Thanks~ The score is really impressive and exciting. Looking forward to your new version and hope we can add the link of this SOTA result to the leaderboard.

Best

yeliudev commented 3 years ago

Hi Yong-Lu,

I've pushed the code for GT boxes to https://github.com/yeliudev/ConsNet. After installing ConsNet from source, simply running the following command will reproduce the results.

python tools/launch.py --config configs/consnet_gt_5e_hico_det.py --checkpoint <path-to-consnet_5e_hico_det-6e89e1d9.pth> --eval

Cheers.