Reconstruction for object further than ±40 cm had no results

ridanlukita commented 2 years ago

Checklist

[X] I have searched for similar issues.
[X] For Python issues, I have tested with the latest development wheel.
[X] I have checked the release documentation and the latest documentation (for master branch).

My Question

Describe the issue

The reconstruction system works great for near objects, however, if the object further than ±40 cm, the results from integrated.ply (in the folder scene) always just a blank, no objects at all.

I record the RGBD (.jpg color and .png depth) image dataset using Intel RealSense D435i with modified MediumDensityPreset from RealSense Viewer

Steps to reproduce the bug

I had done:

In RealSense Viewer : Hardware reset and restore factory calibration, Get 0.00 calibration result from On-Chip Calibration
In realsense.json (configuration file) : changing various voxel_size from 0.05, 0.04, 0.03, 0.02.

realsense.json configuration

{
    "name": "Realsense RGBD file",
    "path_dataset": "dataset/",
    "path_intrinsic": "dataset/camera_intrinsic.json",
    "device":"cuda:0",
    "debug_mode": false,
    "python_multi_threading": true,
    "max_depth": 3.0,
    "min_depth": 0.0,
    "voxel_size": 0.02,
    "max_depth_diff": 0.03,
    "preference_loop_closure_odometry": 0.1,
    "preference_loop_closure_registration": 5.0,
    "tsdf_cubic_size": 3.0,
    "icp_method": "color",
    "method": "slac",
    "global_registration": "ransac"
}

MediumDensityPreset.json

{
    "device": {
        "fw version": "05.13.00.50",
        "name": "Intel RealSense D435I",
        "product line": "D400"
    },
    "parameters": {
        "aux-param-autoexposure-setpoint": "1536",
        "aux-param-colorcorrection1": "0.298828",
        "aux-param-colorcorrection10": "-0",
        "aux-param-colorcorrection11": "-0",
        "aux-param-colorcorrection12": "-0",
        "aux-param-colorcorrection2": "0.293945",
        "aux-param-colorcorrection3": "0.293945",
        "aux-param-colorcorrection4": "0.114258",
        "aux-param-colorcorrection5": "-0",
        "aux-param-colorcorrection6": "-0",
        "aux-param-colorcorrection7": "-0",
        "aux-param-colorcorrection8": "-0",
        "aux-param-colorcorrection9": "-0",
        "aux-param-depthclampmax": "65536",
        "aux-param-depthclampmin": "0",
        "aux-param-disparityshift": "0",
        "controls-autoexposure-auto": "True",
        "controls-autoexposure-manual": "8500",
        "controls-color-autoexposure-auto": "True",
        "controls-color-autoexposure-manual": "166",
        "controls-color-backlight-compensation": "1",
        "controls-color-brightness": "0",
        "controls-color-contrast": "50",
        "controls-color-gain": "64",
        "controls-color-gamma": "300",
        "controls-color-hue": "0",
        "controls-color-power-line-frequency": "1",
        "controls-color-saturation": "64",
        "controls-color-sharpness": "50",
        "controls-color-white-balance-auto": "True",
        "controls-color-white-balance-manual": "4600",
        "controls-depth-gain": "16",
        "controls-laserpower": "360",
        "controls-laserstate": "on",
        "ignoreSAD": "0",
        "param-amplitude-factor": "0.18",
        "param-autoexposure-setpoint": "1536",
        "param-censusenablereg-udiameter": "9",
        "param-censusenablereg-vdiameter": "9",
        "param-censususize": "9",
        "param-censusvsize": "9",
        "param-depthclampmax": "65536",
        "param-depthclampmin": "0",
        "param-depthunits": "100",
        "param-disableraucolor": "0",
        "param-disablesadcolor": "0",
        "param-disablesadnormalize": "0",
        "param-disablesloleftcolor": "0",
        "param-disableslorightcolor": "1",
        "param-disparitymode": "0",
        "param-disparityshift": "0",
        "param-lambdaad": "935",
        "param-lambdacensus": "26",
        "param-leftrightthreshold": "19",
        "param-maxscorethreshb": "887",
        "param-medianthreshold": "1021",
        "param-minscorethresha": "54",
        "param-neighborthresh": "97",
        "param-raumine": "3",
        "param-rauminn": "1",
        "param-rauminnssum": "6",
        "param-raumins": "3",
        "param-rauminw": "5",
        "param-rauminwesum": "11",
        "param-regioncolorthresholdb": "0.0136986",
        "param-regioncolorthresholdg": "0.707436",
        "param-regioncolorthresholdr": "0.181996",
        "param-regionshrinku": "3",
        "param-regionshrinkv": "1",
        "param-robbinsmonrodecrement": "23",
        "param-robbinsmonroincrement": "3",
        "param-rsmdiffthreshold": "1.8125",
        "param-rsmrauslodiffthreshold": "1",
        "param-rsmremovethreshold": "0.482143",
        "param-scanlineedgetaub": "16",
        "param-scanlineedgetaug": "259",
        "param-scanlineedgetaur": "896",
        "param-scanlinep1": "132",
        "param-scanlinep1onediscon": "77",
        "param-scanlinep1twodiscon": "234",
        "param-scanlinep2": "342",
        "param-scanlinep2onediscon": "390",
        "param-scanlinep2twodiscon": "151",
        "param-secondpeakdelta": "600",
        "param-texturecountthresh": "0",
        "param-texturedifferencethresh": "0",
        "param-usersm": "1",
        "param-zunits": "100"
    },
    "schema version": 1,
    "viewer": {
        "stream-depth-format": "Z16",
        "stream-fps": "30",
        "stream-height": "480",
        "stream-width": "848"
    }
}

Error message

There is always warning that appear like this in the step (green text) below

Expected behavior

Longer range reconstruction can be generated

Open3D, Python and System information

- Operating system: Ubuntu 20.04
- Python version: Python 3.8.10
- Open3D version: 0.13.0
- System type: x64
- Is this remote workstation?: no
- How did you install Open3D?: build from source
- Compiler version gcc 9.3.0

Additional information

Around 3 months ago everything works normal, I can make 3D reconstruction of a room, similar to http://www.open3d.org/docs/0.13.0/tutorial/reconstruction_system/integrate_scene.html#results

But around last week until today, I often do scan for near object using mini turntable for 360 degree scanning and changing the disparity shift to around 30-50 and use depth units of 100.

Then I use the camera for scanning long range objects, I change the disparity shift to 0 and depth units to 1000. For color and depth dataset from the camera give no problem, but the reconstruction system give me blank results.

Is there any suggestion to solve these problem?

theNded commented 2 years ago

Depth unit is hard-coded for sensors. If you are still using RealSense D435i, keep using 1000, and do not change it to 100. Otherwise you need to modify every parameter proportionally, including max_depth, voxel_size, max_depth_diff, tsdf_cubic_size, etc.

ridanlukita commented 2 years ago

Thank you! Now I able to do that similar to the tutorial

After I read the documentation for each of the parameter you mentioned, I've understand what each of those parameter used for.

Otherwise you need to modify every parameter proportionally, including max_depth, voxel_size, max_depth_diff, tsdf_cubic_size, etc.

But I want to know further, what/how are the rules for combining these parameters to yield a good reconstruction model?

Is there any guide/tutorial for that?

theNded commented 2 years ago

This is not yet available, but we will update that part once the reconstruction system is fully migrated to the tensor interface. I will close this issue for now and feel free to file other issues if necessary.

isl-org / Open3D