fabienbaradel
commented
1 month ago Hi @slayyden , Thank for your interest in our work. Your synthetic data looks nice! We train Multi-HMR using mostly synthetic data where the FOV is ranging from 50 to 70 degrees and the average FOV is 60 degrees, and maybe 80% of the images have this average FOV. So I guess that Multi-HMR is not very robust to very low of very high FOV, that would be a good extension to do for sure.
So I would say to use the default FOV (60 degrees) or to use your FOVX. The best would be to fine-tune Multi-HMR using your synthetic data with the camera parameters to make the model more rosbut. I hope that my answer helps, Thanks,
I am attempting to run multi-hmr on some synthetic data meant to emulate footage from a front-facing smartphone camera.
Running the demo results in the following image:
If I try to input the fovx of the camera (54 degrees) by modifying the following line in
demo.pyI get the following image:
The silhouette somewhat matches, but the camera is very far back and the rendered face looks flattened compared to the face in the input image. To me, this indicates that the fov is too low.
If I instead input the fovy of the image (85 degrees) I get:
The facial features look no longer flattened, but the model's position is completely off and the camera is still further back than it should be.
Which FOV was the correct one to use? I suspect that the fovy input was correct, but too far from the range of fovs the model was trained on, thus producing a strange output.