search

graphdeco-inria / hierarchical-3d-gaussians

Official implementation of the SIGGRAPH 2024 paper "A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large Datasets"
Other
847 stars 76 forks source link

Questions about network viewer #15

Open stas-polukeev opened 1 month ago

stas-polukeev commented 1 month ago

First of all thanks for the great work on the paper and for publishing your code, the results are very impressive! I have a few questions concerning the network viewer. I tried to find information about network (remote) SIBR viewer in readme, but could not find it.

  1. I have found information only about realtime viewer, does this mean that the remote viewer is not yet supported for hierarchical 3d gs, as the process of rasterization is different from the original 3d gs?
  2. In the sceneario, where I have these components: A - my local laptop (ubuntu) B - the remote machine (ubuntu) with gpus, where the training is being done C - the docker container on B I use to actually run the training script Is it possible to view the training process or the results of training in C using the resources of B to view the rendered images on A (my laptop)? If it is possible, how should I approach this? Thanks in advance!
Snosixtyboo commented 1 month ago

Hi, thanks for the question,

actually, the 3DGS remote viewer should be compatible with all 3 stages (coarse, single, post).

What you are describing should definitely be possible, but not trivial. You would need to start the remote server on C, and then forward the X window from C -> B -> A, where you need an X window client. It's likely there will be some challenges for starting an OpenGL application on C.

In theory, there should be a much easier solution, where you just port forward the used --port from C -> B -> A, and start the remote viewer on A. In practice, I think there will be some issues if the remote viewer doesn't find the input files on A. If I have some time I can check if we can adapt the remote viewer to support that, but I will be pretty busy the next weeks.

Snosixtyboo commented 1 month ago

Hi, thanks for the question,

actually, the 3DGS remote viewer should be compatible with all 3 stages (coarse, single, post).

What you are describing should definitely be possible, but not trivial. You would need to start the remote server on C, and then forward the X window from C -> B -> A, where you need an X window client. It's likely there will be some challenges for starting an OpenGL application on C.

In theory, there should be a much easier solution, where you just port forward the used --port from C -> B -> A, and start the remote viewer on A. In practice, I think there will be some issues if the remote viewer doesn't find the input files on A. If I have some time I can check if we can adapt the remote viewer to support that, but I will be pretty busy the next weeks.

Actually, this might just work. Making sure port (default is 6009) is forwarded from C to A, and then having a copy of the COLMAP directory (with "sparse" inside it) locally, and passing a path to that directory to the remote viewer at startup, via --path.

stas-polukeev commented 1 month ago

@Snosixtyboo thanks for your response! I'll dig into this, and if I find some working solution, I could share it

stas-polukeev commented 1 month ago

@Snosixtyboo I have managed to launch SIBR as I wanted using x server in both docker and on remote server, while using ssh -X to connect to remote, however it crushes immediately with the error Segmentation fault (core dump) while also throwing "Cameras ID inconsistent. Setting default interpolation path." error. I've tested some other default gui apps such as firefox for example, and it works fine, so the problem is not X server. I suppose there are some problems with remote launch of opengl apps with X server. Could you may be provide some insight or tips, where can I investigate futher? any ideas would be welcome, thank you!