we develop a polarization imaging method for 3D reconstruction combined with fused coarse depth from speckle stereo to achieve reconstruction of less-texture object with highly reflective region. The contributions of this study can be summarized as follows: 1) We propose a polarization 3D imaging method that does not depend on the texture of objects, which intuitively fuses coarse depth with polarization information by establishing a large linear sparse equation system; 2) We discuss the necessity of taking into account both diffuse and specular reflection types under the perspective projection model
The first step involves deriving polarization normals from polarized images. This process utilizes the intensity differences at various polarization angles to calculate the orientation of the surface normals based on the polarization effect. (As shown in step1_decompose_polarisation_6.m)
The second step calculates a rough depth map from the polarization data. Based on different transmission projection models, this step also solves for surface guided normals, which are crucial for accurate surface orientation and depth estimation. (As shown in step2_guide_normal_orthogonal.m and step2_guide_normal_perspective.m)
In step three, the surface guided normals are used to disambiguate the polarization normals. This step is essential for resolving any inconsistencies in normal directions derived from the polarized images. (As shown in step3_NormalCorrection folder)
The final step merges the rough depth map with the disambiguated polarization normals. This fusion process enhances the accuracy and quality of the final surface reconstruction, leading to more precise and detailed 3D models. (step4_merge_depth_normal.m)
sudo apt-get install libopengm-bin libopengm-dev libopengm-doc python-opengm python-opengm-docHere's how to run each step of the process:
step1.m and step2.m to process the initial stages.step4.m to complete the fusion of data.The Figure_AO_paper folder in the current directory contains visualization results and demos. To view the demo, navigate to this folder and follow the instructions provided within to run the visualization scripts.
Due to the size limit on file uploads, the data required to run the demos is stored on Google Drive. You can access and download the necessary files from the following link:
Please ensure you have downloaded and placed the data in the appropriate directory as described in the visualization scripts before running the demos.
The Utils folder contains multiple utility functions that are essential for the various processing steps outlined above. These utilities provide additional functionality, such as data handling and transformations, that support the main scripts. Be sure to check this folder for any necessary functions that may need to be included or referenced in your script executions.
Chuheng Xu, Yao Hu
This project builds upon ideas and code from the following research papers:
D. Zhu and W.A.P. Smith, "Depth from a polarisation + RGB stereo pair"
Ye Yu and William A. P. Smith, "Depth estimation meets inverse rendering for single image novel view synthesis"