search

zhuhao-nju / facescape

FaceScape (PAMI2023 & CVPR2020)
833 stars 94 forks source link

camera intrinsics and extrinsics #90

Closed X-niper closed 1 year ago

X-niper commented 2 years ago

Hi, thanks for sharing this work and the high-quality dataset!

You share the original multi-view camera figure together with the TU-model. I found that the camera intrinsics and extrinsics you provide are different between different scanning. The question is that, the camera parameters, which are in param.json file, are inferred from multi-view photogrammetry or measured/provided according to the hardware.

I think it's more reasonable to provide the original camera parameters that are determined by your setup because they can be used in multi-view photogrammetry. Then, if they are provided according to the hardware, why they differ between different scanning?

Your response are much appreciated!

zhuhao-nju commented 2 years ago

Hi @X-niper ,

We captured the data using multiple hardware setups, which are located in two places for the convenience to capture different people. For different setups, the position and pose of the cameras have been adjusted, so we don't have an uniform intrinsic/extrinsic parameters for all the tuples of images.

In the 3D reconstruction phase, we used structure-from-motion to solve intrinsic and extrinsic parameters for each tuple of images, which can be understood as self-calibration. So the intrinsic/extrinsic parameters obtained are different. We provide global rotation/translation and scale parameters for each tuple to align them in a canonical space (https://github.com/zhuhao-nju/facescape/blob/master/toolkit/demo_align.ipynb), but they are only approximately aligned.

X-niper commented 2 years ago

Hi, @zhuhao-nju So glad to hear your response! I found that camera parameters are different even for scans that belong to one person but different expressions . Does this mean that the parameters you provide are self-calibrated? Again, than you so much for this work and your response:>

X-niper commented 2 years ago

Another question is as follows. To get multi view model (.ply file) from multi-view data (many .jpg files), do you use commercial software such as Metashape or your own tools ?

zhuhao-nju commented 2 years ago

Hi @X-niper , Yes, these parameters are obtained by self-calibration that is the same thing as structure-from-motion mentioned earlier. We used our own tools, while Metashape should work as well, both of which are based on structure-from-motion and multi-view stereo algorithms.