How can we evaluate the model with Known Object Mode?

[JacobYuan7]

Thanks for your great work, but I am bit confused about the Known Object Evaluation mode. Is is possible to release the code for this?

[yoyomimi]

Please refer this script: https://github.com/YueLiao/PPDM/blob/master/src/lib/eval/set_hico_evalution.sh

[JacobYuan7]

Please refer this script: https://github.com/YueLiao/PPDM/blob/master/src/lib/eval/set_hico_evalution.sh

Thanks for your reply. I have tried this code but find the results all zeros. Any tips on how to reproduce the results in the paper?

[yoyomimi]

Please refer this script: https://github.com/YueLiao/PPDM/blob/master/src/lib/eval/set_hico_evalution.sh

Thanks for your reply. I have tried this code but find the results all zeros. Any tips on how to reproduce the results in the paper?

Hello, the eval tool is a little bit complex, you need to check every process of the evaluation gets valid results. If you can't locate your problem, you could give a detail description about how you use the eval tools so we can help you better.

[JacobYuan7]

Please refer this script: https://github.com/YueLiao/PPDM/blob/master/src/lib/eval/set_hico_evalution.sh

Thanks for your reply. I have tried this code but find the results all zeros. Any tips on how to reproduce the results in the paper?

Hello, the eval tool is a little bit complex, you need to check every process of the evaluation gets valid results. If you can't locate your problem, you could give a detail description about how you use the eval tools so we can help you better.

This is what I have done:

1. I save HOI prediction results generated from the model ( ranked by verb score ) as best_predictions.json. (Currently I am trying this model). The basic data structure is [{'predictions': bboxes, 'hoi_predictions': hois, 'file_name': XXX}...] The object category indices range from 1 to 90 and verb category indices range from 1 to 117.

2. Then I do as this file suggests _set_hicoevalution.sh

3. Finally, I launch MATLAB to run _Generatedetection.m. Then I get all zeros.

Is it caused by the model I use? Thank you so much for your patience.

[yoyomimi]

Please refer this script: https://github.com/YueLiao/PPDM/blob/master/src/lib/eval/set_hico_evalution.sh

Thanks for your reply. I have tried this code but find the results all zeros. Any tips on how to reproduce the results in the paper?

Hello, the eval tool is a little bit complex, you need to check every process of the evaluation gets valid results. If you can't locate your problem, you could give a detail description about how you use the eval tools so we can help you better.

This is what I have done:

1. I save HOI prediction results generated from the model ( ranked by verb score ) as best_predictions.json. (Currently I am trying this model). The basic data structure is [{'predictions': bboxes, 'hoi_predictions': hois, 'file_name': XXX}...] The object category indices range from 1 to 90 and verb category indices range from 1 to 117.
2. Then I do as this file suggests _set_hicoevalution.sh
3. Finally, I launch MATLAB to run _Generatedetection.m. Then I get all zeros.

Is it caused by the model I use? Thank you so much for your patience.

Hello, sorry for the late reply.

If you cannot get the reported results (eval the best_predictions.json using the provided evaluation code) before transfer the result format, firstly make sure you have successfully loaded the model (e.g. the saved model path can not be found), you can check your eval.log. Also, the model you used has a different structure from us, you should make sure that all the model params in the files correctly load the corresponding values in the checkpoint.

If you can get the right results as reported, then you can run the _set_hicoevalution.sh to transfer the results format and eval on known objects. If you still get all zeros, the way you run the shell might be wrong. This needs more patience to see each output through the format transform process and locate the problem.