Closed kermitfrog closed 2 years ago
Hi, I spent quit a while checking this issue. This tool aims to converting rendered depth and color images to underwater for robotic simulation platforms, when utilizing real world depth maps, the imperfect depth maps will cause several artifacts which require additional pre-processing.
The attached files you uploaded somehow have different image sizes with the original middlebury data, i uploaded the tool in another repo: https://github.com/omv-ysong/DSV/blob/93f79302607fd10f5d0a77cf768881cf1545cf6c/tools/convert_disparity_to_depth.py You can use it to generate .exr depth map. Since it still contains empty depth value, i add an depth inpainting function in the renderer, you can turn on it in the yaml config file (refine_depth: true).
This is the test result:
using following config parameters: scale_factor: 4.0 scale_factor_bs: 8000.0 render_back_scatter: true volumetric_max_depth: 8.0 num_volumetric_slabs: 8 write_slab: false slab_sampling_method: 1 image_width_height: [2632, 1988] camera_intrinsic_matrix: [7315.238, 0.0, 997.555, 0.0, 7315.238, 980.754, 0.0, 0.0, 1.0] white_balance: [2.28613, 1.0, 1.361328] water_attenuation_RGB: [0.37, 0.044, 0.035] vsf_type: 3 light_spectrum_RGB: [0.25, 0.35, 0.4] light_ambient_RGB: [0.4, 0.4, 0.4] light_RID_type: 1 num_lights: 1 light_positions_XYZ: [[0.5, 0.0, 0.0]] light_orientations_RollPitchYaw: [[0.0, 0.0, 0.0]] write_uw_img_in_exr: false depth_sommoth_window_size: 5 refine_depth: true auto_iso: false
Close this issue since no further comments.
Background: I am trying to generate images for evaluation of depth-from-stereo algorithms for a deep sea environment. Since there are none I am aware of, I'm trying to make deep sea versions of the Middlebury dataset (and MPI Sintel).
I am encountering 2-3 types of Rendering bugs in the DeepSeaRenderer, as shown in the attached picture (based on "Jadeplant" from the 2014 Middlebury dataset).
Red: happening in ConvertDoubleMatTo8Bit() due to negative values not beeing checked for and causing an underflow.
Blue: "Shadows" caused by small differences in depth of neighboring pixels (> ~0.01). Does look unnatural..
Green: Rough edges. caused by bigger differences in depth of neighboring pixels (> ~0.1). Does look very unnatural..
Red can be fixed by adding the following lines to ConvertDoubleMatTo8Bit():
I also noticed that the auto_iso setting is ignored in 2 of 3 versions of this function.
Blue and Green may have the same cause, but I don't know how to improve it. I tried blurring the depth map, which mostly takes care of Blue, but makes Green look even worse. I noticed these bugs seem not to happen in your testdata. Do you have any ideas why?
I tried various configuration parameters, starting with the config file from the testdata and several pictures from the Middlebury and MPI Sintel datasets and had these bugs everywhere (with one exception that did not have Red).
I tried attaching the image and my computed depth map (as exr (hope that saved right..) and as .pfm -- which is what Middlebury uses.), but as that didn't work, I uploaded it here: https://aguzinski.de/BWPubq6/Jadeplant.zip