Dense_slam_gui.py couldn't excute my own dataset

[JomanaAshraf]

Checklist

• [X] I have searched for similar issues.
• [X] For Python issues, I have tested with the latest development wheel.
• [X] I have checked the release documentation and the latest documentation (for master branch).

My Question

Hii, I am trying to use the dense_slam_gui.py with my dataset which is captured with intel realsense D435 and I stored the color and depth images and also the camera intrinsics as required then I create a config file and named it as "realsense_config.yml"as shown here:

name: Default reconstruction system config
fragment_size: 100
device: CUDA:0
engine: tensor
multiprocessing: false
path_dataset: '/home/jomana/masters_ws/src/Studying/Thesis/reconstruction_folder'
depth_folder: depth
color_folder: color
path_intrinsic: '/home/jomana/masters_ws/src/Studying/Thesis/reconstruction_folder/camera_intrinsic.json'
depth_min: 0.1
depth_max: 3.0
depth_scale: 1000.0
odometry_method: hybrid
odometry_loop_interval: 10
odometry_loop_weight: 0.1
odometry_distance_thr: 0.07
icp_method: colored
icp_voxelsize: 0.05
icp_distance_thr: 0.07
global_registration_method: ransac
registration_loop_weight: 0.1
integration_mode: color
voxel_size: 0.0058
trunc_voxel_multiplier: 8.0
block_count: 40000
surface_weight_thr: 3.0

but when I excute the code by running this command:

python dense_slam_gui.py --config /home/jomana/masters_ws/src/Studying/Thesis/Open3D/examples/python/t_reconstruction_system/realsense_config.yml --device "CPU:0"

It gives me this error as the depth and color images are not seen by the code:

File "dense_slam_gui.py", line 331, in update_main depth_ref = o3d.t.io.read_image(depth_file_names[0]) IndexError: list index out of range

I tried to print the number of files but it gives me 0 so can you tell me what is the problem with this and why It couldn't process my own dataset?

I used the latest release of the repo

[Kareemziada]

Based on the error message you provided, it looks like the dense_slam_gui.py script is not able to find any depth images in your dataset. Here are some potential solutions you could try:

Verify that the path to the dataset folder is correct: Make sure that the path specified in the path_dataset parameter in your config file is the correct path to the folder containing the color and depth images.

Verify that the file names match the expected format: The code assumes that the color and depth images have file names in the format color/{fragment_id}/{frame_id}.jpg and depth/{fragment_id}/{frame_id}.png, respectively, where {fragment_id} and {frame_id} are integers. Make sure that your file names match this format.

Check that the file extensions are correct: Make sure that your color images are in JPEG format and your depth images are in PNG format. The code assumes these file extensions.

Verify that the intrinsic file is correct: Make sure that the path to the camera intrinsics file specified in the path_intrinsic parameter in your config file is correct and that the file contains the correct intrinsic parameters for your camera.

Check that the depth images are readable: Try opening one of the depth images in a program like ImageJ or Fiji to verify that it can be read and that it contains valid depth information.

Verify that the fragment_size parameter is set correctly: If your dataset contains fewer frames than the fragment_size parameter, the code will not be able to find any depth images. Make sure that the fragment_size parameter is set to a value that is less than or equal to the number of frames in your dataset.

If none of these suggestions solve the issue, please provide more details about your dataset or share a sample of your data so we can further investigate the problem. Let us know if you have any questions or need further assistance.

[JomanaAshraf]

Thanks a lot for answering me and I saw that the problem was from my dataset as there were redundent photos in the depth images so the number of depth and color was not equal so this was the problem.

I have another question, When I tried to use it with my dataset which is apple planatation dataset I got this results which is not good but I was using CPU is the results will be good enough if using CUDA also, what should I do to get good results?

[Kareemziada]

Improving the quality of a reconstruction can be challenging, but there are several things you can try to improve your results.

Firstly, using a GPU (such as with CUDA) can speed up the reconstruction process, especially for large datasets. However, keep in mind that the quality of the reconstruction is also dependent on the quality of the input data.

Here are some tips that might help improve your results: Ensure good lighting conditions: Make sure that your scene is well-lit and that there are no strong shadows or reflections that could interfere with the depth estimation.

Optimize camera placement: Try to capture images from multiple angles and positions to ensure good coverage of the scene. Also, make sure that the camera is properly calibrated and that the intrinsic parameters are accurate.

Choose appropriate reconstruction parameters: The parameters you choose for the reconstruction process can have a big impact on the quality of the final result. For example, choosing a smaller voxel size can result in a more detailed reconstruction, but may also be more computationally intensive.

Use high-quality depth data: If possible, try to capture depth data using a high-quality sensor. For example, the Intel RealSense D455 is a newer model that provides improved depth accuracy over the D435.

Use multiple sensors: Consider using multiple cameras or sensors to capture data from different viewpoints. This can help to improve the coverage and quality of the reconstruction.

[JomanaAshraf]

Thanks for your reply @Kareemziada

I tried to use CUDA in the reconstruction but I got this error:

RuntimeError: [Open3D Error] (void open3d::core::__OPEN3D_CUDA_CHECK(cudaError_t, const char*, int)) /home/jomana/masters_ws/src/Studying/Thesis/Open3D/cpp/open3d/core/CUDAUtils.cpp:288: /home/jomana/masters_ws/src/Studying/Thesis/Open3D/cpp/open3d/core/MemoryManagerCUDA.cpp:23 CUDA runtime error: out of memory

When I try to increase the voxel size it works ok but the reconstruction is not good. I need to decrease the voxel size not to increase it. So why it gives me this error and how can I fix it?