Is it possible to inject a camera position before the solve of the human pose ?

shubham-goel / 4D-Humans

4DHumans: Reconstructing and Tracking Humans with Transformers

https://shubham-goel.github.io/4dhumans/

MIT License

1.25k stars 120 forks source link

Is it possible to inject a camera position before the solve of the human pose ? #112

Closed hbardak closed 6 months ago

hbardak commented 6 months ago

Hi guys,

Firstly, amazing work ! I am a VFX artist, that have a bit of coding knowledge. in the VFX industry, usually we have already solve with great precision the camera position in the 3D world ( translation, rotation, focal, lens distortion). Would you see a way to pass a solved camera to human 4D to get a better accuracy for the human poses ?

At the moment I got :

Which obviously doesn't work for me.

Any ideas how to fix it ?

geopavlakos commented 6 months ago

I'm not sure exactly how this visualization was produced, but the first suggestion I have is to use the actual focal length value of the camera (by modifying the scaled_focal_length value of the demo). Please take a look at this issue too. If that looks better for each frame independently, then it should be easy to apply the rotation and translation of each frame to the meshes of the corresponding frame.