puzzlepaint / surfelmeshing

Real-time surfel-based mesh reconstruction from RGB-D video.
BSD 3-Clause "New" or "Revised" License
420 stars 83 forks source link

patch_match_stereo CUDA for pinhole camera #16

Open tiziano-v opened 4 years ago

tiziano-v commented 4 years ago

Hi there! I'm trying to use patch_match_stereo in CUDA to compute my depth map from a pair of frames captured using a stereo camera (pinhole camera). I managed to get some results from the code, but actually they seem just the texture of the rigth frame that i'm giving as input and not the correct overlap between the 2 frames. Could you please give me some explanation about the code and how to tune/use correctly the parameters and the inputs? Currently i'm setting: MatchMetric matchmetric = MatchMetric::kZNCC; int contextradius = 2; float min_initialdepth = 0.0022f; float max_initialdepth = 20.0f; int iterationcount = 30; float max_normal_2dlength = 2.0f;

My inputs are: left and rigth frames from calibrated stereo camera; fx, fy, cx, cy for Pin-hole camera parameters; The roto-translation matrix between the 2 camera and its inverse; CUDAUnprojectionLookup2D reference_unprojection (stereo_camera, cudaStream);

If you need something else i'll provide it immediately. Thank you very much!

puzzlepaint commented 4 years ago

Does this have to do with surfelmeshing? I actually don't know whether the patch match stereo code in libvis works in the version of libvis which is in this repository, I guess it shouldn't be used anywhere in this application unless I forgot about it?

However, in the version of libvis in the camera_calibration repository, it should definitely work (although by default it is compiled to use a generic camera model). See stereo_depth_estimation.cc for an example of how to use it. In the float2 Project(float3 position) function in pixel_corner_projector.cuh, the camera model is hard-coded in this version. There are lots of commented-out sections of the code for various camera models. The pinhole model seems to be right at the start.