search

graphdeco-inria / hierarchical-3d-gaussians

Official implementation of the SIGGRAPH 2024 paper "A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large Datasets"
Other
959 stars 93 forks source link

Nothing is displayed form the SIBR viewer #20

Open kevintsq opened 3 months ago

kevintsq commented 3 months ago

Ubuntu 20.04, CUDA 12.1

~/hierarchical-3d-gaussians/SIBR_viewers/install/bin$ ./SIBR_gaussianHierarchyViewer_app --path ~/test/camera_calibration/aligned --scaffold ~/test/output/scaffold/point_cloud/iteration_30000 --model-path ~/test/output/merged.hier --images-path  ~/test/camera_calibration/rectified/images
[SIBR] --  INFOS  --:   Initialization of GLFW
[SIBR] --  INFOS  --:   OpenGL Version: 4.6.0 NVIDIA 530.30.02[major: 4, minor: 6]
Number of input Images to read: 3015
Number of Cameras set up: 3015
[SIBR] --  INFOS  --:   Error: can't load mesh '/home/kevin/test/camera_calibration/aligned/sparse/0/points3d.txt.
[SIBR] --  INFOS  --:   Mesh contains: colors: 1, normals: 1, texcoords: 0
[SIBR] --  INFOS  --:   Mesh '/home/kevin/test/camera_calibration/aligned/sparse/0/points3d.ply successfully loaded. 1 meshes were loaded with a total of  (0) faces and  (787346) vertices detected. Init GL ...
[SIBR] --  INFOS  --:   Init GL mesh complete 
 USED RES (854,480) scene w h 854 : 480 NAME 0567.jpg
Warning: GLParameter user_color does not exist in shader PointBased
[SIBR] --  INFOS  --:   Allowing up to 16720112 Gaussians in VRAM
Using 11434796440 bytes for scene
[SIBR] --  INFOS  --:   Initializing Raycaster
[SIBR] --  INFOS  --:   Interactive camera using (0.009,1100) near/far planes.
Warning: GLParameter user_color does not exist in shader points_shader
Switched to trackball mode.
images_path: /home/kevin/test/camera_calibration/rectified/images
[SIBR] --  INFOS  --:   Deinitialization of GLFW

image The directory is as follows:

~/test$ tree --filelimit 100
.
├── camera_calibration
│   ├── aligned
│   │   ├── chunks.txt
│   │   └── sparse
│   │       └── 0
│   │           ├── cameras.bin
│   │           ├── images.bin
│   │           ├── points3D.bin
│   │           └── points3d.ply
│   ├── chunks
│   │   ├── 0_0
│   │   │   ├── center.txt
│   │   │   ├── extent.txt
│   │   │   └── sparse
│   │   │       └── 0
│   │   │           ├── cameras.bin
│   │   │           ├── depth_params.json
│   │   │           ├── images.bin
│   │   │           ├── points3D.bin
│   │   │           └── points3D.ply
│   │   ├── 1_0
│   │   │   ├── center.txt
│   │   │   ├── extent.txt
│   │   │   └── sparse
│   │   │       └── 0
│   │   │           ├── cameras.bin
│   │   │           ├── depth_params.json
│   │   │           ├── images.bin
│   │   │           ├── points3D.bin
│   │   │           └── points3D.ply
│   │   ├── 2_0
│   │   │   ├── center.txt
│   │   │   ├── extent.txt
│   │   │   └── sparse
│   │   │       └── 0
│   │   │           ├── cameras.bin
│   │   │           ├── depth_params.json
│   │   │           ├── images.bin
│   │   │           ├── points3D.bin
│   │   │           └── points3D.ply
│   │   ├── 3_0
│   │   │   ├── center.txt
│   │   │   ├── extent.txt
│   │   │   └── sparse
│   │   │       └── 0
│   │   │           ├── cameras.bin
│   │   │           ├── depth_params.json
│   │   │           ├── images.bin
│   │   │           ├── points3D.bin
│   │   │           └── points3D.ply
│   │   ├── 4_0
│   │   │   ├── center.txt
│   │   │   ├── extent.txt
│   │   │   └── sparse
│   │   │       └── 0
│   │   │           ├── cameras.bin
│   │   │           ├── depth_params.json
│   │   │           ├── images.bin
│   │   │           ├── points3D.bin
│   │   │           └── points3D.ply
│   │   └── 5_0
│   │       ├── center.txt
│   │       ├── extent.txt
│   │       └── sparse
│   │           └── 0
│   │               ├── cameras.bin
│   │               ├── depth_params.json
│   │               ├── images.bin
│   │               └── points3D.bin
│   ├── raw_chunks
│   │   ├── 0_0
│   │   │   ├── bundle_adjustment
│   │   │   │   ├── database.db
│   │   │   │   ├── images  [128 entries exceeds filelimit, not opening dir]
│   │   │   │   ├── matching_50.txt
│   │   │   │   ├── run-colmap-geometric.sh
│   │   │   │   ├── run-colmap-photometric.sh
│   │   │   │   ├── sparse
│   │   │   │   │   ├── 0
│   │   │   │   │   │   ├── cameras.bin
│   │   │   │   │   │   ├── images.bin
│   │   │   │   │   │   └── points3D.bin
│   │   │   │   │   ├── b
│   │   │   │   │   ├── cameras.bin
│   │   │   │   │   ├── images.bin
│   │   │   │   │   ├── o
│   │   │   │   │   │   ├── cameras.bin
│   │   │   │   │   │   ├── images.bin
│   │   │   │   │   │   └── points3D.bin
│   │   │   │   │   ├── points3D.bin
│   │   │   │   │   ├── t
│   │   │   │   │   └── t2
│   │   │   │   └── stereo
│   │   │   │       ├── consistency_graphs
│   │   │   │       ├── depth_maps
│   │   │   │       ├── fusion.cfg
│   │   │   │       ├── normal_maps
│   │   │   │       └── patch-match.cfg
│   │   │   ├── center.txt
│   │   │   ├── extent.txt
│   │   │   ├── log.err
│   │   │   ├── log.out
│   │   │   └── sparse
│   │   │       └── 0
│   │   │           ├── cameras.bin
│   │   │           ├── images.bin
│   │   │           ├── matching_50.txt
│   │   │           └── points3D.bin
│   │   ├── 1_0
│   │   │   ├── bundle_adjustment
│   │   │   │   ├── database.db
│   │   │   │   ├── images  [687 entries exceeds filelimit, not opening dir]
│   │   │   │   ├── matching_50.txt
│   │   │   │   ├── run-colmap-geometric.sh
│   │   │   │   ├── run-colmap-photometric.sh
│   │   │   │   ├── sparse
│   │   │   │   │   ├── 0
│   │   │   │   │   │   ├── cameras.bin
│   │   │   │   │   │   ├── images.bin
│   │   │   │   │   │   └── points3D.bin
│   │   │   │   │   ├── b
│   │   │   │   │   ├── cameras.bin
│   │   │   │   │   ├── images.bin
│   │   │   │   │   ├── o
│   │   │   │   │   │   ├── cameras.bin
│   │   │   │   │   │   ├── images.bin
│   │   │   │   │   │   └── points3D.bin
│   │   │   │   │   ├── points3D.bin
│   │   │   │   │   ├── t
│   │   │   │   │   └── t2
│   │   │   │   └── stereo
│   │   │   │       ├── consistency_graphs
│   │   │   │       ├── depth_maps
│   │   │   │       ├── fusion.cfg
│   │   │   │       ├── normal_maps
│   │   │   │       └── patch-match.cfg
│   │   │   ├── center.txt
│   │   │   ├── extent.txt
│   │   │   ├── log.err
│   │   │   ├── log.out
│   │   │   └── sparse
│   │   │       └── 0
│   │   │           ├── cameras.bin
│   │   │           ├── images.bin
│   │   │           ├── matching_50.txt
│   │   │           └── points3D.bin
│   │   ├── 2_0
│   │   │   ├── bundle_adjustment
│   │   │   │   ├── database.db
│   │   │   │   ├── images  [812 entries exceeds filelimit, not opening dir]
│   │   │   │   ├── matching_50.txt
│   │   │   │   ├── run-colmap-geometric.sh
│   │   │   │   ├── run-colmap-photometric.sh
│   │   │   │   ├── sparse
│   │   │   │   │   ├── 0
│   │   │   │   │   │   ├── cameras.bin
│   │   │   │   │   │   ├── images.bin
│   │   │   │   │   │   └── points3D.bin
│   │   │   │   │   ├── b
│   │   │   │   │   ├── cameras.bin
│   │   │   │   │   ├── images.bin
│   │   │   │   │   ├── o
│   │   │   │   │   │   ├── cameras.bin
│   │   │   │   │   │   ├── images.bin
│   │   │   │   │   │   └── points3D.bin
│   │   │   │   │   ├── points3D.bin
│   │   │   │   │   ├── t
│   │   │   │   │   └── t2
│   │   │   │   └── stereo
│   │   │   │       ├── consistency_graphs
│   │   │   │       ├── depth_maps
│   │   │   │       ├── fusion.cfg
│   │   │   │       ├── normal_maps
│   │   │   │       └── patch-match.cfg
│   │   │   ├── center.txt
│   │   │   ├── extent.txt
│   │   │   ├── log.err
│   │   │   ├── log.out
│   │   │   └── sparse
│   │   │       └── 0
│   │   │           ├── cameras.bin
│   │   │           ├── images.bin
│   │   │           ├── matching_50.txt
│   │   │           └── points3D.bin
│   │   ├── 3_0
│   │   │   ├── bundle_adjustment
│   │   │   │   ├── database.db
│   │   │   │   ├── images  [887 entries exceeds filelimit, not opening dir]
│   │   │   │   ├── matching_50.txt
│   │   │   │   ├── run-colmap-geometric.sh
│   │   │   │   ├── run-colmap-photometric.sh
│   │   │   │   ├── sparse
│   │   │   │   │   ├── 0
│   │   │   │   │   │   ├── cameras.bin
│   │   │   │   │   │   ├── images.bin
│   │   │   │   │   │   └── points3D.bin
│   │   │   │   │   ├── b
│   │   │   │   │   ├── cameras.bin
│   │   │   │   │   ├── images.bin
│   │   │   │   │   ├── o
│   │   │   │   │   │   ├── cameras.bin
│   │   │   │   │   │   ├── images.bin
│   │   │   │   │   │   └── points3D.bin
│   │   │   │   │   ├── points3D.bin
│   │   │   │   │   ├── t
│   │   │   │   │   └── t2
│   │   │   │   └── stereo
│   │   │   │       ├── consistency_graphs
│   │   │   │       ├── depth_maps
│   │   │   │       ├── fusion.cfg
│   │   │   │       ├── normal_maps
│   │   │   │       └── patch-match.cfg
│   │   │   ├── center.txt
│   │   │   ├── extent.txt
│   │   │   ├── log.err
│   │   │   ├── log.out
│   │   │   └── sparse
│   │   │       └── 0
│   │   │           ├── cameras.bin
│   │   │           ├── images.bin
│   │   │           ├── matching_50.txt
│   │   │           └── points3D.bin
│   │   ├── 4_0
│   │   │   ├── bundle_adjustment
│   │   │   │   ├── database.db
│   │   │   │   ├── images  [756 entries exceeds filelimit, not opening dir]
│   │   │   │   ├── matching_50.txt
│   │   │   │   ├── run-colmap-geometric.sh
│   │   │   │   ├── run-colmap-photometric.sh
│   │   │   │   ├── sparse
│   │   │   │   │   ├── 0
│   │   │   │   │   │   ├── cameras.bin
│   │   │   │   │   │   ├── images.bin
│   │   │   │   │   │   └── points3D.bin
│   │   │   │   │   ├── b
│   │   │   │   │   ├── cameras.bin
│   │   │   │   │   ├── images.bin
│   │   │   │   │   ├── o
│   │   │   │   │   │   ├── cameras.bin
│   │   │   │   │   │   ├── images.bin
│   │   │   │   │   │   └── points3D.bin
│   │   │   │   │   ├── points3D.bin
│   │   │   │   │   ├── t
│   │   │   │   │   └── t2
│   │   │   │   └── stereo
│   │   │   │       ├── consistency_graphs
│   │   │   │       ├── depth_maps
│   │   │   │       ├── fusion.cfg
│   │   │   │       ├── normal_maps
│   │   │   │       └── patch-match.cfg
│   │   │   ├── center.txt
│   │   │   ├── extent.txt
│   │   │   ├── log.err
│   │   │   ├── log.out
│   │   │   └── sparse
│   │   │       └── 0
│   │   │           ├── cameras.bin
│   │   │           ├── images.bin
│   │   │           ├── matching_50.txt
│   │   │           └── points3D.bin
│   │   └── 5_0
│   │       ├── bundle_adjustment
│   │       │   ├── database.db
│   │       │   ├── images  [311 entries exceeds filelimit, not opening dir]
│   │       │   ├── matching_50.txt
│   │       │   ├── run-colmap-geometric.sh
│   │       │   ├── run-colmap-photometric.sh
│   │       │   ├── sparse
│   │       │   │   ├── 0
│   │       │   │   │   ├── cameras.bin
│   │       │   │   │   ├── images.bin
│   │       │   │   │   └── points3D.bin
│   │       │   │   ├── b
│   │       │   │   ├── cameras.bin
│   │       │   │   ├── images.bin
│   │       │   │   ├── o
│   │       │   │   │   ├── cameras.bin
│   │       │   │   │   ├── images.bin
│   │       │   │   │   └── points3D.bin
│   │       │   │   ├── points3D.bin
│   │       │   │   ├── t
│   │       │   │   └── t2
│   │       │   └── stereo
│   │       │       ├── consistency_graphs
│   │       │       ├── depth_maps
│   │       │       ├── fusion.cfg
│   │       │       ├── normal_maps
│   │       │       └── patch-match.cfg
│   │       ├── center.txt
│   │       ├── extent.txt
│   │       ├── log.err
│   │       ├── log.out
│   │       └── sparse
│   │           └── 0
│   │               ├── cameras.bin
│   │               ├── images.bin
│   │               ├── matching_50.txt
│   │               └── points3D.bin
│   └── rectified
│       ├── depths  [3015 entries exceeds filelimit, not opening dir]
│       ├── images  [3015 entries exceeds filelimit, not opening dir]
│       └── sparse
│           ├── 0
│           │   ├── cameras.bin
│           │   ├── images.bin
│           │   └── points3D.bin
│           ├── model
│           │   ├── cameras.txt
│           │   ├── images.txt
│           │   └── points3D.txt
│           └── orig
│               ├── 0
│               │   ├── cameras.bin
│               │   ├── images.bin
│               │   ├── points3D.bin
│               │   └── project.ini
│               └── text
│                   ├── cameras.txt
│                   ├── images.txt
│                   └── points3D.txt
└── output
    ├── merged.hier
    ├── scaffold
    │   ├── cameras.json
    │   ├── cfg_args
    │   ├── exposure.json
    │   ├── input.ply
    │   └── point_cloud
    │       └── iteration_30000
    │           ├── pc_info.txt
    │           └── point_cloud.ply
    └── trained_chunks
        ├── 0_0
        │   ├── anchors.bin
        │   ├── cameras.json
        │   ├── cfg_args
        │   ├── exposure.json
        │   ├── hierarchy.hier
        │   ├── hierarchy.hier_opt
        │   ├── input.ply
        │   └── point_cloud
        │       ├── iteration_15000
        │       └── iteration_30000
        │           ├── pc_info.txt
        │           └── point_cloud.ply
        ├── 1_0
        │   ├── anchors.bin
        │   ├── cameras.json
        │   ├── cfg_args
        │   ├── exposure.json
        │   ├── hierarchy.hier
        │   ├── hierarchy.hier_opt
        │   ├── input.ply
        │   └── point_cloud
        │       ├── iteration_15000
        │       └── iteration_30000
        │           ├── pc_info.txt
        │           └── point_cloud.ply
        ├── 2_0
        │   ├── anchors.bin
        │   ├── cameras.json
        │   ├── cfg_args
        │   ├── exposure.json
        │   ├── hierarchy.hier
        │   ├── hierarchy.hier_opt
        │   ├── input.ply
        │   └── point_cloud
        │       ├── iteration_15000
        │       └── iteration_30000
        │           ├── pc_info.txt
        │           └── point_cloud.ply
        ├── 3_0
        │   ├── anchors.bin
        │   ├── cameras.json
        │   ├── cfg_args
        │   ├── exposure.json
        │   ├── hierarchy.hier
        │   ├── hierarchy.hier_opt
        │   ├── input.ply
        │   └── point_cloud
        │       ├── iteration_15000
        │       └── iteration_30000
        │           ├── pc_info.txt
        │           └── point_cloud.ply
        ├── 4_0
        │   ├── anchors.bin
        │   ├── cameras.json
        │   ├── cfg_args
        │   ├── exposure.json
        │   ├── hierarchy.hier
        │   ├── hierarchy.hier_opt
        │   ├── input.ply
        │   └── point_cloud
        │       ├── iteration_15000
        │       └── iteration_30000
        │           ├── pc_info.txt
        │           └── point_cloud.ply
        └── 5_0
            ├── anchors.bin
            ├── cameras.json
            ├── cfg_args
            ├── exposure.json
            ├── hierarchy.hier
            ├── hierarchy.hier_opt
            ├── input.ply
            └── point_cloud
                ├── iteration_15000
                └── iteration_30000
                    ├── pc_info.txt
                    └── point_cloud.ply

152 directories, 258 files
stas-polukeev commented 3 months ago

I have a similar problem with the viewer, have you found a solution?

kevintsq commented 3 months ago

Unfortunately not yet.

leilei0330 commented 3 months ago

I have the same issue under Windows

Snosixtyboo commented 3 months ago

Hi,

thanks for bringing it up. We will try to look into it. I can think of two possible reasons that are not yet caught by error handling:

1) OpenGL/CUDA Interop is not working. Do these issues happen on a multi-GPU machine, or a laptop? 2) The memory budget set for the GPU is too high. If your GPU has less than 24 GB of RAM, you can try setting -budget to, e.g. 70% of your total VRAM memory (in MB, e.g., for 6 GB budget, use --budget 6000).

leilei0330 commented 3 months ago

image image image

Thank you very much for your reply, here are some of the issues I've had, I'd appreciate it if you could take a look:

  1. I think it may be the first reason you said, but if it's this problem, I don't know how to solve it, because I tested it on a single graphics card, and I used 4090.
  2. Here are some of my screenshots in Interp. mode, most of the time it's all black, sometimes these spots appear.
  3. When I turn on "show sfm", I can see the point cloud, so the gaussian is not rendered.
  4. I don't have two small windows related to top viewe in my SIBR
kevintsq commented 3 months ago

Hi,

thanks for bringing it up. We will try to look into it. I can think of two possible reasons that are not yet caught by error handling:

  1. OpenGL/CUDA Interop is not working. Do these issues happen on a multi-GPU machine, or a laptop?
  2. The memory budget set for the GPU is too high. If your GPU has less than 24 GB of RAM, you can try setting -budget to, e.g. 70% of your total VRAM memory (in MB, e.g., for 6 GB budget, use --budget 6000).

Adding --budget 6000 works! Thanks!

I'm using a laptop that has been configured in the BIOS to use the dedicated GPU only. If I use the mix mode (integrated GPU + dedicated GPU), it will show a segmentation fault. After attaching a gdb to it, it shows that the error comes from something like libcuda.

Snosixtyboo commented 3 months ago

Thanks for the responses. Regarding the issues on some systems: CUDA/OpenGL interop is required for high performance, and not all systems support it, unfortunately. We will try to add some guidance to the program to ensure it's as easy to activate as possible and provide a slower fallback on systems where it absolutely can't run, just like we did for the original 3DGS.

leilei0330 commented 3 months ago

Thanks for the responses. Regarding the issues on some systems: CUDA/OpenGL interop is required for high performance, and not all systems support it, unfortunately. We will try to add some guidance to the program to ensure it's as easy to activate as possible and provide a slower fallback on systems where it absolutely can't run, just like we did for the original 3DGS.

Thank you very much for your hard work and look forward to your updates

haofengsiji commented 2 months ago

same problem here, docker env: colmap/colmap unbuntu 22.04, pytorch 2.3.0+cu121, nvcc 12.1, GTX3080 10GB.

--budget not work.

Is there any other method can solve this problem ? for example, change a docker env, etc...

gosick-victorique commented 1 month ago

image image image

Thank you very much for your reply, here are some of the issues I've had, I'd appreciate it if you could take a look:

  1. I think it may be the first reason you said, but if it's this problem, I don't know how to solve it, because I tested it on a single graphics card, and I used 4090.
  2. Here are some of my screenshots in Interp. mode, most of the time it's all black, sometimes these spots appear.
  3. When I turn on "show sfm", I can see the point cloud, so the gaussian is not rendered.
  4. I don't have two small windows related to top viewe in my SIBR

Hello, I have encountered the same problem as you, may I ask if you have solved these problems

leilei0330 commented 1 month ago

image image image Thank you very much for your reply, here are some of the issues I've had, I'd appreciate it if you could take a look:

  1. I think it may be the first reason you said, but if it's this problem, I don't know how to solve it, because I tested it on a single graphics card, and I used 4090.
  2. Here are some of my screenshots in Interp. mode, most of the time it's all black, sometimes these spots appear.
  3. When I turn on "show sfm", I can see the point cloud, so the gaussian is not rendered.
  4. I don't have two small windows related to top viewe in my SIBR

Hello, I have encountered the same problem as you, may I ask if you have solved these problems

Sorry I didn't. I still could not render the scene.