search

muskie82 / MonoGS

[CVPR'24 Highlight & Best Demo Award] Gaussian Splatting SLAM
https://rmurai.co.uk/projects/GaussianSplattingSLAM/
Other
1.3k stars 117 forks source link

Errors while trying demo #80

Closed hnglp closed 1 month ago

hnglp commented 5 months ago

When I try to run the demo, I get the following output and encounter two different error messages. No matter how long I wait, there is no change in the image, and it eventually stops on its own. I couldn't install the libjpeg library with conda as mentioned in the second error group, but currently, libjpeg-turbo, libnvjpeg, and libpng are installed. Still, I don't know why I'm getting this error. If you have any suggestions for solutions to these two error groups, please share them with me. My system is Windows 11, and I'm wondering if using WSL could be a solution to these errors.

image 
Traceback (most recent call last):
  File "C:\Programms\anaconda3\envs\MonoGS\lib\multiprocessing\process.py", line 314, in _bootstrap
    self.run()
  File "C:\Programms\anaconda3\envs\MonoGS\lib\multiprocessing\process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\Unreal User\Desktop\alp_akbaci_masterstudent\github_projects\MonoGS\utils\slam_backend.py", line 380, in run
    self.map(self.current_window)
  File "C:\Users\Unreal User\Desktop\alp_akbaci_masterstudent\github_projects\MonoGS\utils\slam_backend.py", line 171, in map
    render_pkg = render(
  File "C:\Users\Unreal User\Desktop\alp_akbaci_masterstudent\github_projects\MonoGS\gaussian_splatting\gaussian_renderer\__init__.py", line 66, in render
    projmatrix=viewpoint_camera.full_proj_transform,
  File "C:\Users\Unreal User\Desktop\alp_akbaci_masterstudent\github_projects\MonoGS\utils\camera_utils.py", line 101, in full_proj_transform
    self.world_view_transform.unsqueeze(0).bmm(
  File "C:\Users\Unreal User\Desktop\alp_akbaci_masterstudent\github_projects\MonoGS\utils\camera_utils.py", line 96, in world_view_transform
    return getWorld2View2(self.R, self.T).transpose(0, 1)
  File "C:\Users\Unreal User\Desktop\alp_akbaci_masterstudent\github_projects\MonoGS\gaussian_splatting\utils\graphics_utils.py", line 42, in getWorld2View2
    C2W = torch.linalg.inv(Rt)
torch._C._LinAlgError: linalg.inv: The diagonal element 1 is zero, the inversion could not be completed because the input matrix is singular.
Traceback (most recent call last):
  File "C:\Users\Unreal User\Desktop\alp_akbaci_masterstudent\github_projects\MonoGS\slam.py", line 252, in <module>
    slam = SLAM(config, save_dir=save_dir)
  File "C:\Users\Unreal User\Desktop\alp_akbaci_masterstudent\github_projects\MonoGS\slam.py", line 110, in __init__
    self.frontend.run()
  File "C:\Users\Unreal User\Desktop\alp_akbaci_masterstudent\github_projects\MonoGS\utils\slam_frontend.py", line 392, in run
    render_pkg = self.tracking(cur_frame_idx, viewpoint)
  File "C:\Users\Unreal User\Desktop\alp_akbaci_masterstudent\github_projects\MonoGS\utils\slam_frontend.py", line 164, in tracking
    render_pkg = render(
  File "C:\Users\Unreal User\Desktop\alp_akbaci_masterstudent\github_projects\MonoGS\gaussian_splatting\gaussian_renderer\__init__.py", line 65, in render
    viewmatrix=viewpoint_camera.world_view_transform,
  File "C:\Users\Unreal User\Desktop\alp_akbaci_masterstudent\github_projects\MonoGS\utils\camera_utils.py", line 96, in world_view_transform
    return getWorld2View2(self.R, self.T).transpose(0, 1)
  File "C:\Users\Unreal User\Desktop\alp_akbaci_masterstudent\github_projects\MonoGS\gaussian_splatting\utils\graphics_utils.py", line 42, in getWorld2View2
    C2W = torch.linalg.inv(Rt)
C:\Users\Unreal User\AppData\Roaming\Python\Python310\site-packages\torchvision\io\image.py:13: UserWarning: Failed to load image Python extension: '[WinError 127] The specified procedure could not be found'If you don't plan on using image functionality from `torchvision.io`, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have `libjpeg` or `libpng` installed before building `torchvision` from source?
  warn(
FEngine (64 bits) created at 0000027532131050 (threading is enabled)
FEngine resolved backend: OpenGL
C:\Users\Unreal User\AppData\Roaming\Python\Python310\site-packages\torchvision\io\image.py:13: UserWarning: Failed to load image Python extension: '[WinError 127] The specified procedure could not be found'If you don't plan on using image functionality from `torchvision.io`, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have `libjpeg` or `libpng` installed before building `torchvision` from source?
  warn(
MonoGS: Resetting the system
MonoGS: Initialized map
Process Process-3:
Traceback (most recent call last):
File "C:\Programms\anaconda3\envs\MonoGS\lib\multiprocessing\process.py", line 314, in _bootstrap
    self.run()
File "C:\Programms\anaconda3\envs\MonoGS\lib\multiprocessing\process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
File "C:\Users\Unreal User\Desktop\alp_akbaci_masterstudent\github_projects\MonoGS\utils\slam_backend.py", line 380, in run
    self.map(self.current_window)
File "C:\Users\Unreal User\Desktop\alp_akbaci_masterstudent\github_projects\MonoGS\utils\slam_backend.py", line 183, in map
    render_pkg["render"],
TonyDua commented 5 months ago

I also have the same issue as you. An error occurred in the first demo demonstration.Looking forward to the author's reply and hoping to discuss the follow-up progress with you. Thank you.


(MonoGS) PS C:\AI_Workstation\MonoGS> python slam.py --config configs/mono/tum/fr3_office.yaml
MonoGS: saving results in results\datasets_tum\2024-04-29-11-20-00
FEngine (64 bits) created at 000001899F2C8050 (threading is enabled)
FEngine resolved backend: OpenGL
MonoGS: Resetting the system
MonoGS: Initialized map
Traceback (most recent call last):
  File "C:\AI_Workstation\MonoGS\slam.py", line 252, in <module>
    slam = SLAM(config, save_dir=save_dir)
  File "C:\AI_Workstation\MonoGS\slam.py", line 110, in __init__
    self.frontend.run()
  File "C:\AI_Workstation\MonoGS\utils\slam_frontend.py", line 392, in run
    render_pkg = self.tracking(cur_frame_idx, viewpoint)
  File "C:\AI_Workstation\MonoGS\utils\slam_frontend.py", line 164, in tracking
    render_pkg = render(
  File "C:\AI_Workstation\MonoGS\gaussian_splatting\gaussian_renderer\__init__.py", line 65, in render
    viewmatrix=viewpoint_camera.world_view_transform,
  File "C:\AI_Workstation\MonoGS\utils\camera_utils.py", line 96, in world_view_transform
    return getWorld2View2(self.R, self.T).transpose(0, 1)
  File "C:\AI_Workstation\MonoGS\gaussian_splatting\utils\graphics_utils.py", line 41, in getWorld2View2
    C2W = torch.linalg.inv(Rt)
torch._C._LinAlgError: linalg.inv: The diagonal element 2 is zero, the inversion could not be completed because the input matrix is singular.
MonoGS: Resetting the opacity of non-visible Gaussians
MonoGS: Resetting the opacity of non-visible Gaussians
MonoGS: Resetting the opacity of non-visible Gaussians
MonoGS: Resetting the opacity of non-visible Gaussians

in void __cdecl filament::FCamera::setProjection(enum filament::Camera::Projection,double,double,double,double,double,double) noexcept:89
reason: Camera preconditions not met. Using default projection.

C:\AI_Workstation\MonoGS\gui\slam_gui.py:507: RuntimeWarning: invalid value encountered in scalar divide
  2 * np.arctan(width * np.tan(np.deg2rad(vfov_deg) / 2) / height)
Process Process-4:
Traceback (most recent call last):
  File "C:\ProgramData\anaconda3\envs\MonoGS\lib\multiprocessing\process.py", line 315, in _bootstrap
    self.run()
  File "C:\ProgramData\anaconda3\envs\MonoGS\lib\multiprocessing\process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "C:\AI_Workstation\MonoGS\gui\slam_gui.py", line 688, in run
    app.run()
  File "C:\AI_Workstation\MonoGS\gui\slam_gui.py", line 676, in update
    self.scene_update()
  File "C:\AI_Workstation\MonoGS\gui\slam_gui.py", line 663, in scene_update
    self.render_gui()
  File "C:\AI_Workstation\MonoGS\gui\slam_gui.py", line 654, in render_gui
    current_cam = self.get_current_cam()
  File "C:\AI_Workstation\MonoGS\gui\slam_gui.py", line 525, in get_current_cam
    current_cam = Camera.init_from_gui(
  File "C:\AI_Workstation\MonoGS\utils\camera_utils.py", line 87, in init_from_gui
    projection_matrix = getProjectionMatrix2(
  File "C:\AI_Workstation\MonoGS\gaussian_splatting\utils\graphics_utils.py", line 74, in getProjectionMatrix2
    left = ((2 * cx - W) / W - 1.0) * W / 2.0
ZeroDivisionError: division by zero
MonoGS: Resetting the opacity of non-visible Gaussians
MonoGS: Resetting the opacity of non-visible Gaussians
MonoGS: Resetting the opacity of non-visible Gaussians
MonoGS: Resetting the opacity of non-visible Gaussians
muskie82 commented 5 months ago

Hi, As discussed in #20, this seems to be an issue with pytorch multiprocess on Windows. We never tested the system on Windons and we don't have Windows OS locally. Some people reported the bug in PyTorch official repo, but no one seems to find the solution.

So a possible approach is to build Ubuntu virtual environment on docker and run the method.

Best,

Choujim commented 4 months ago

it seems to be ans issue with pytorch mp lib, i set single_thread: True in ./config/YOURconfig.yaml, the program runs normally.

hnglp commented 4 months ago

I had previously set up my virtual environment on Windows and i get the error. So, after @muskie82's previous response, I created an environment on WSL/Ubuntu and tried to run the project there.

After your suggestion, I adjusted the 'single thread' part in both systems, but I'm still getting the following errors. I believe it's due to a matrix singularity issue. I would greatly appreciate your help with this.

Win11:

User
Traceback (most recent call last):
  File "C:\Users\Unreal User\Desktop\alp_akbaci_masterstudent\github_projects\MonoGS\slam.py", line 252, in <module>
    slam = SLAM(config, save_dir=save_dir)
  File "C:\Users\Unreal User\Desktop\alp_akbaci_masterstudent\github_projects\MonoGS\slam.py", line 110, in __init__
    self.frontend.run()
  File "C:\Users\Unreal User\Desktop\alp_akbaci_masterstudent\github_projects\MonoGS\utils\slam_frontend.py", line 392, in run
    render_pkg = self.tracking(cur_frame_idx, viewpoint)
  File "C:\Users\Unreal User\Desktop\alp_akbaci_masterstudent\github_projects\MonoGS\utils\slam_frontend.py", line 164, in tracking
    render_pkg = render(
  File "C:\Users\Unreal User\Desktop\alp_akbaci_masterstudent\github_projects\MonoGS\gaussian_splatting\gaussian_renderer\__init__.py", line 65, in render
    viewmatrix=viewpoint_camera.world_view_transform,
  File "C:\Users\Unreal User\Desktop\alp_akbaci_masterstudent\github_projects\MonoGS\utils\camera_utils.py", line 96, in world_view_transform
    return getWorld2View2(self.R, self.T).transpose(0, 1)
  File "C:\Users\Unreal User\Desktop\alp_akbaci_masterstudent\github_projects\MonoGS\gaussian_splatting\utils\graphics_utils.py", line 41, in getWorld2View2
    C2W = torch.linalg.inv(Rt)
torch._C._LinAlgError: linalg.inv: The diagonal element 1 is zero, the inversion could not be completed because the input matrix is singular.

WSL/Ubuntu 20.04:

User
Traceback (most recent call last):
  File "/home/unreal_user/anaconda3/envs/MonoGS/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/home/unreal_user/anaconda3/envs/MonoGS/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/home/unreal_user/MonoGS/utils/slam_backend.py", line 406, in run
    self.initialize_map(cur_frame_idx, viewpoint)
  File "/home/unreal_user/MonoGS/utils/slam_backend.py", line 90, in initialize_map
    viewpoint, self.gaussians, self.pipeline_params, self.background
  File "/home/unreal_user/MonoGS/gaussian_splatting/gaussian_renderer/__init__.py", line 66, in render
    projmatrix=viewpoint_camera.full_proj_transform,
  File "/home/unreal_user/MonoGS/utils/camera_utils.py", line 102, in full_proj_transform
    self.projection_matrix.unsqueeze(0)
RuntimeError: CUDA error: CUBLAS_STATUS_NOT_SUPPORTED when calling `cublasSgemmStridedBatched( handle, opa, opb, m, n, k, &alpha, a, lda, stridea, b, ldb, strideb, &beta, c, ldc, stridec, num_batches)`

Also, if you'd like, you can take a look at another issue I have here.

hnglp commented 1 month ago

I solved my problem by running on Linux machine