Inaccurate Predictions and Inconsistencies in opal23_headpose_test.py

[jayong-sv]

Hello, I recently renamed the file "example.tif" to "example_bak.tif" to exclude its annotation file (example.json) and ran the script "opal23_headpose_test.py". However, I encountered significant variations in the predicted values depending on the dataset and rotation mode, resulting in inaccurate predictions.

I observed the following:

• The predicted values varied substantially based on the dataset used.
• Different rotation modes led to inconsistent results.
• The overall accuracy of the predictions was poor.

I am seeking guidance on how to obtain more accurate and consistent predictions from the "opal23_headpose_test.py" script. Any suggestions or insights into improving the prediction accuracy would be greatly appreciated.

• 300wlp_euler

  

• 300wlp_6d

  

• panoptic_6d

  

• panoptic_6d_opal

  

• panoptic_euler

  

[alejandro-cobo]

Hello, thank you for your interest.

If I understood correctly, you are trying to call opal23_headpose_test.py without providing the example.json annotations, right? Since the model needs bounding box annotations to crop the face, I assume you downloaded and configured this face detection repository to get the bounding box annotations. The code defaults to this repo if the JSON file is not found.

As for the wrong predictions you are getting, the models trained on 300W-LP should output the correct predictions. I could not reproduce your results even without the example.json annotations. The models trained on CMU Panoptic, however, won't work on in-the-wild images because the dataset only includes one specific type of background / illumination.

Let me know if this answered your questions.

[jayong-sv]

Thank you for your response! Yes, I used the SSD detector by default. The 6D predicted values for 300W-LP are accurate, but the 300W-LP Euler predicted values differ from the 300W-LP 6D values. I noted that Panoptic may not perform well on in-the-wild images. Thanks.

[alejandro-cobo]

That's strange, I get the following results using SSD detector and 300wlp_euler model:

And with 300wlp_6d:

Maybe you have an old version of the repo? Have you tried running git pull on opal23_headpose and images_framework repos?

[jayong-sv]

I set up opal23_headpose and images_framework today, so they should be the latest versions. I really appreciate you taking the time to reproduce the test for me. Given the results, it seems I need to investigate further to determine if there are any issues with my code (I added SSD, images_framework, and updated some paths). Thanks again for your help!