SSH connection problem

[Vonjor]

Bothering you again. I want to train on the server, so I connect to the server via vs code's ssh, and I run this command:

./scripts/process_and_train_sai_custom.sh ./myvideo/spectacular-rec_2024-05-31_11-02-35.zip

it reported the following errors:

(nerfstudio) root@autodl:~/autodl-tmp/3dgs-deblur# ./scripts/process_and_train_sai_custom.sh ./myvideo/spectacular-rec_2024-05-31_11-02-35.zip ++ basename ./myvideo/spectacular-rec_2024-05-31_11-02-35.zip

• NAME_W_EXT=spectacular-rec_2024-05-31_11-02-35.zip
• NAME=spectacular-rec_2024-05-31_11-02-35
• : ON
• : OFF
• '[' ON == ON ']'
• PREVIEW_FLAG=--preview
• '[' OFF == ON ']'
• COLMAP_FLAG=
• python process_sai_custom.py ./myvideo/spectacular-rec_2024-05-31_11-02-35.zip --preview COMMAND: ['unzip', './myvideo/spectacular-rec_2024-05-31_11-02-35.zip', '-d', '/tmp/tmpxa8fype5/recording'] Archive: ./myvideo/spectacular-rec_2024-05-31_11-02-35.zip inflating: /tmp/tmpxa8fype5/recording/data.mp4
  inflating: /tmp/tmpxa8fype5/recording/data.jsonl
  inflating: /tmp/tmpxa8fype5/recording/calibration.json
  inflating: /tmp/tmpxa8fype5/recording/vio_config.yaml
  COMMAND: ['sai-cli', 'process', '/tmp/tmpxa8fype5/recording', '/tmp/tmp8v4ugk7w/sai-cli/spectacular-rec_2024-05-31_11-02-35', '--image_format=png', '--no_undistort', '--key_frame_distance=0.1', '--internal=maxKeypoints:2000', '--internal=optimizerMaxIterations:50', '--preview', '--preview3d'] Selected device type: android Traceback (most recent call last): File "/root/miniconda3/envs/nerfstudio/bin/sai-cli", line 8, in sys.exit(main()) File "/root/miniconda3/envs/nerfstudio/lib/python3.8/site-packages/spectacularAI/cli/sai_cli.py", line 15, in main args.func(args) File "/root/miniconda3/envs/nerfstudio/lib/python3.8/site-packages/spectacularAI/cli/process/process.py", line 719, in process from spectacularAI.cli.visualization.visualizer import Visualizer, VisualizerArgs File "/root/miniconda3/envs/nerfstudio/lib/python3.8/site-packages/spectacularAI/cli/visualization/visualizer.py", line 7, in from OpenGL.GL import # all prefixed with gl so OK to import File "/root/miniconda3/envs/nerfstudio/lib/python3.8/site-packages/OpenGL/GL/init.py", line 4, in from OpenGL.GL.VERSION.GL_1_1 import File "/root/miniconda3/envs/nerfstudio/lib/python3.8/site-packages/OpenGL/GL/VERSION/GL_1_1.py", line 14, in from OpenGL.raw.GL.VERSION.GL_1_1 import File "/root/miniconda3/envs/nerfstudio/lib/python3.8/site-packages/OpenGL/raw/GL/VERSION/GL_1_1.py", line 7, in from OpenGL.raw.GL import _errors File "/root/miniconda3/envs/nerfstudio/lib/python3.8/site-packages/OpenGL/raw/GL/_errors.py", line 4, in _error_checker = _ErrorChecker( _p, _p.GL.glGetError ) AttributeError: 'NoneType' object has no attribute 'glGetError' Traceback (most recent call last): File "process_sai_custom.py", line 140, in process(args) File "process_sai_custom.py", line 102, in process maybe_run_cmd(cmd) File "process_sai_custom.py", line 36, in maybe_run_cmd if not args.dry_run: subprocess.check_call(cmd) File "/root/miniconda3/envs/nerfstudio/lib/python3.8/subprocess.py", line 364, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['sai-cli', 'process', '/tmp/tmpxa8fype5/recording', '/tmp/tmp8v4ugk7w/sai-cli/spectacular-rec_2024-05-31_11-02-35', '--image_format=png', '--no_undistort', '--key_frame_distance=0.1', '--internal=maxKeypoints:2000', '--internal=optimizerMaxIterations:50', '--preview', '--preview3d']' returned non-zero exit status 1.

Therefore, I want to know whether the sai-cli window cannot be opened through ssh，I tried Xvfb and it didn't work either

[oseiskar]

There's an environment variable that can fix this: use

SAI_PREVIEW=OFF ./scripts/process_and_train_sai_custom.sh ...

[Vonjor]

This is my fault, I did not read the.sh file carefully, thank you for your answer

[Vonjor]

After training with process_and_train_sai_custom.sh, I found that the rolling shutter processing is not very good, and I found this.sh file mentioned that it does not currently support rolling shutter and motion blur compensation at the same time

This version uses both motion blur compensation only and should work well with iPhone data and other devices with short rolling shutter readout times (or global shutter cameras)

Will there be implementations that can compensate for both motion blur and rolling shutter? Although it is not possible to compensate at the same time for the time being, I have compared with the benchmark 3DGS, and your project has indeed been greatly improved, and it is indeed a great job!

[oseiskar]

process_and_train_sai_custom.sh already has both motion blur and rolling shutter compensation, but it only works with Android data, where the readout time is known (sorry, the comment in the script is a bit unclear). However, there is no guarantee that it would always work well.