search

graphdeco-inria / gaussian-splatting

Original reference implementation of "3D Gaussian Splatting for Real-Time Radiance Field Rendering"
https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/
Other
14.5k stars 1.89k forks source link

3D rendering using SIBR viewer #892

Open absiddique11 opened 3 months ago

absiddique11 commented 3 months ago

Can anyone help me with how to render 3D video/pointcloud using SIBR viewer?

jaco001 commented 3 months ago

Viewer from this project (as I know) is for static GS from INRIA. With small hack you can inport any GS. For 3D video you must rebuild viewer for original paper for 3d video.

absiddique11 commented 3 months ago

Viewer from this project (as I know) is for static GS from INRIA. With small hack you can inport any GS. For 3D video you must rebuild viewer for original paper for 3d video.

Thank you. Pardon my ignorance, could you tell me the steps I need to follow to export 3D views?

jaco001 commented 3 months ago

image

  1. Hit record
  2. Click in main window
  3. Move your camera with keyboard (keys wasd and ijkl or gimbal)
  4. Hit stop image
  5. Check save video from play
  6. Hit play and watch if this is ok for you. image
  7. Export video

GL.

absiddique11 commented 3 months ago

image

  1. Hit record
  2. Click in main window
  3. Move your camera with keyboard (keys wasd and ijkl or gimbal)
  4. Hit stop image
  5. Check save video from play
  6. Hit play and watch if this is ok for you. image
  7. Export video

GL.

Thank you but is it possible to render as 3D views where we can also visualize the depth information? I mean (x, y and z). Current rendering video seem 2D having x and y.

jaco001 commented 3 months ago

Depth infromation you can utilize at any level cuz splats are a ply file with extra info. Use eg. meshlab https://stackoverflow.com/questions/42754843/ply-polygonal-mesh-to-2d-depth-map

For 3D scene render i will be utilize Unity or Unreal Engine with plugins for splats ... and try this way render 3D/stereo video Blender plugin is rather for cleaning then proper rendering... but you can try too.

absiddique11 commented 3 months ago

Depth infromation you can utilize at any level cuz splats are a ply file with extra info. Use eg. meshlab https://stackoverflow.com/questions/42754843/ply-polygonal-mesh-to-2d-depth-map

For 3D scene render i will be utilize Unity or Unreal Engine with plugins for splats ... and try this way render 3D/stereo video Blender plugin is rather for cleaning then proper rendering... but you can try too.

Thank you so much. Could you explain a little bit more how I can use unity or unreal engine for the trained model using Gaussian splatting algorithm to render 3d/stereo video? I can see trained model has point-cloud files for input, 7000 iterations and 30000 iterations.

jaco001 commented 3 months ago

Unreal Engine plugin for splats https://github.com/xverse-engine/XV3DGS-UEPlugin (there are other too)

360 rendering tutorial https://www.youtube.com/watch?v=SpkP1XCF_mE

absiddique11 commented 3 months ago

Unreal Engine plugin for splats https://github.com/xverse-engine/XV3DGS-UEPlugin (there are other too)

360 rendering tutorial https://www.youtube.com/watch?v=SpkP1XCF_mE

Thank you so much.