search

postech-ami / FastMETRO

[ECCV'22] Official PyTorch Implementation of "Cross-Attention of Disentangled Modalities for 3D Human Mesh Recovery with Transformers"
https://fastmetro.github.io/
MIT License
164 stars 14 forks source link

Demo with object detector and project to the original image #19

Closed mimiliaogo closed 1 year ago

mimiliaogo commented 1 year ago

Hi, I used an off-the-shelf object detector and get the bounding box of the input image. I sent the bounding box into the model, but how could I project the mesh back to the original input image? I could only project the mesh back to the bounding box image, but if I project back to the original image, the ratio is incorrect. Is there any formula for converting the camera parameters or anything else I can do?

Thank you!

Youwang-Kim commented 1 year ago

Hi, thank you for your interest for our work.

If you have a cropped information (bbox corner info, etc...), you can estimate the transformation matrix from your original image to the cropped image. (You can refer to https://github.com/yfeng95/DECA/blob/a11554ae2a2b0f3998cf1fa94dd4db03babb34a2/decalib/datasets/vox.py#L98)

Then, you can project the mesh images back to the original resolution by simply inverting that transformation matrix. You need to store the transformation matrix when you perform crop operation and need to invert it to get inverse transformation matrix.

Please reopen this issue if you need more help regarding this.