search

karol-202 / direct-3dgs-segmentation

Semantic 3D segmentation of 3D Gaussian Splats
MIT License
5 stars 1 forks source link

Visualization Problem #1

Closed dingxusen closed 3 months ago

dingxusen commented 3 months ago

Hello, I have completed the training, but I don’t know how to visualize the model. Can you help me write out the visualization process? I look forward to your reply.

karol-202 commented 3 months ago

Hi, it's great to hear that you find my repository useful. I did the visualization by loading/composing a 3DGS scene, segmenting it using a previously trained model, resaving as a .ply file and opening with the SIBR viewer from https://github.com/graphdeco-inria/gaussian-splatting. The visualize.py script in this repo does it, but it will for sure need some adjustments because I did not have the time to properly parametrize it via command-line arguments. These are some things that you will for sure need to change to make the script work for you:

After these modifications, running the script should generate a .ply file, which you should be able to open with the SIBR viewer.

If you have any further, questions/problems, feel free to ask.

dingxusen commented 3 months ago

Hi, it's great to hear that you find my repository useful. I did the visualization by loading/composing a 3DGS scene, segmenting it using a previously trained model, resaving as a .ply file and opening with the SIBR viewer from https://github.com/graphdeco-inria/gaussian-splatting. The visualize.py script in this repo does it, but it will for sure need some adjustments because I did not have the time to properly parametrize it via command-line arguments. These are some things that you will for sure need to change to make the script work for you:

  • specifying the path to the 3DGS models dataset (the same as the one used for training) DATA_PATH = '...'
  • specifying which models exactly you want to be in the visualized scene
models = [
    GaussianModel.load_from('/3dgs-dataset/bed/bed_0001/point_cloud/iteration_15000/point_cloud.ply', EXTRA_FEATURES_TO_LOAD).normalized().with_label(CLASS2LABEL['bed']),
    GaussianModel.load_from('/3dgs-dataset/toilet/toilet_0001/point_cloud/iteration_15000/point_cloud.ply', EXTRA_FEATURES_TO_LOAD).normalized().with_label(CLASS2LABEL['toilet']),
    GaussianModel.load_from('/3dgs-dataset/chair/chair_0003/point_cloud/iteration_15000/point_cloud.ply', EXTRA_FEATURES_TO_LOAD).normalized().with_label(CLASS2LABEL['chair']),
]
  • specifying the colors in CLASSES2COLORS if you need other classes
  • specifying the name of the log folder (experiment name) you used for training (should be a directory in ./log/sem_seg):
args = dotdict({
    ...
    'log_dir': '<EXPERIMENT_NAME>',
    ...
})
  • specifying the output path (for the .ply file): colored_model.save_ply('<PATH>/point_cloud/iteration_15000/point_cloud.ply', include_normals=True)
  • supplying rest of the files the SIBR viewer needs next to the generated .ply file (e.g. cameras.json, etc.)

After these modifications, running the script should generate a .ply file, which you should be able to open with the SIBR viewer.

If you have any further, questions/problems, feel free to ask.

Thank you for your very detailed answer, which is very helpful to me.