search

facebookresearch / phosa

Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wild
Other
177 stars 21 forks source link

How can I reproduce the results in Table 1? #20

Closed monacv closed 3 years ago

monacv commented 3 years ago

I am interested in reproducing the results in Table 1. Could you please guide me what are the steps in doing so? So far I have been able to run the demo on some images and get sensible results.

Screenshot from 2021-03-15 18-48-08

jasonyzhang commented 3 years ago

Hi,

I just posted the code to run the web interface for our forced choice evaluation here: https://github.com/jasonyzhang/annotator

The evaluation is qualitative, so you (or someone else) will need to determine preference between two different images at a time.

To generate the images for the independent composition, you can run the existing code as is for 0 iterations. To generate the images for ours (which is what each row gets compared with), you can run the existing code for the full number of iterations. To generate the images corresponding to each ablation, set that loss term to 0, and then run the existing code for the full number of iterations.

The filenames of the generated images should begin with the image id.

The COCO test set ids for the images used for the evaluation are here.

I am also happy to email you the rendered images that were used in the evaluation (the contains of the ablations folder for the annotator)

Best, Jason

monacv commented 3 years ago

Thank you so much. This was very valuable to know.