search

Anttwo / SuGaR

[CVPR 2024] Official PyTorch implementation of SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering
https://anttwo.github.io/sugar/
Other
2.36k stars 182 forks source link

NO Dgs7000_sdfestim02_sdfnorm02_level03_decim1000000.ply NOR ./output/refined_ply// DIRECTORY CREATED #113

Open nbmontasufv opened 10 months ago

nbmontasufv commented 10 months ago

Hi, thanks for sharing your work with everybody:

I have an issue concerning the ".ply" and "refined_ply" directory generation at the end of the training (on Windows machine):

Command => python train.py -s gaussian_splatting\data\templos\01 -c gaussian_splatting\output\ -r sdf --eval "True" --refinement_time "short" --export_ply "True" -t "true

Error message: RPly: Unable to open file [Open3D WARNING] Read PLY failed: unable to open file: ./output/coarse_mesh\gaussian_splatting\data\templos\01\sugarmesh_coarse\gaussian_splatting\data\templos\01\3Dgs7000_sdfestim02_sdfnorm02_level03_decim1000000.ply Mesh to bind to loaded. Binding radiance cloud to surface mesh... Traceback (most recent call last): File "D:\sugar_local_clone\SuGaR\train.py", line 170, in refined_sugar_path = refined_training(refined_args) File "D:\sugar_local_clone\SuGaR\sugar_trainers\refine.py", line 396, in refined_training sugar = SuGaR( File "D:\sugar_local_clone\SuGaR\sugar_scene\sugar_model.py", line 323, in init scales = scales.clamp_min(0.0000001).reshape(len(faces_verts), -1, 1).expand(-1, self.n_gaussians_per_surface_triangle, 2).clone().reshape(-1, 2) RuntimeError: cannot reshape tensor of 0 elements into shape [0, -1, 1] because the unspecified dimension size -1 can be any value and is ambiguous

Thank you for any heads up concerning this issue.

Best,

Nelson

Anttwo commented 10 months ago

Hello @nbmontasufv,

Thank you for your nice words!

Are you using Windows? Unfortunately, the code currently encounters some issues with Windows. The problem is a convention for writing paths: Windows uses \ as a separator in paths but Linux and MacOS use /. I plan to fix this very soon, I'm really sorry for this.

(In the meantime, if you like to run research code from GitHub --often written for Linux-- on your computer, you can take a look at WSL2 (Windows Subsystem for Linux v2). WSL2 is a free and great tool from Microsoft that lets you use a Linux console in your Windows OS. I use WSL2 on my personal Windows machine, and it really makes life simpler for running research code built for Linux and for developers experimenting with computer science stuff in general)

Best, Antoine

nbmontasufv commented 10 months ago

Hi, Thank you for your quick response and great work. Your modeling and code writing is phenomenal.

Yes, I'm trying to run it on windows. I saw a similar problem in another issue and I'm trying to fix it right now: I tried to change the .split("/") and "./output/coarse"; "/ouput/meshes" etc. but have not been successful to bypass it. I'll get back to you if I find a solution.

Again, thanks so much for your wonderful work and getting back to me.

citystrawman commented 10 months ago

Hi, Thank you for your quick response and great work. Your modeling and code writing is phenomenal.

Yes, I'm trying to run it on windows. I saw a similar problem in another issue and I'm trying to fix it right now: I tried to change the .split("/") and "./output/coarse"; "/ouput/meshes" etc. but have not been successful to bypass it. I'll get back to you if I find a solution.

Again, thanks so much for your wonderful work and getting back to me.

HI, I just encountered a similar issue, I've updated the os separator, but from the code I could only see that sugar_mesh_path is created, but the corresponding .ply is not, and then the code starts loading the mesh by o3d.io.read_triangle_mesh(sugar_mesh_path), which is impossible. So may I know how did you solved this issue?

nbmontasufv commented 10 months ago

Hi, Thank you for your quick response and great work. Your modeling and code writing is phenomenal. Yes, I'm trying to run it on windows. I saw a similar problem in another issue and I'm trying to fix it right now: I tried to change the .split("/") and "./output/coarse"; "/ouput/meshes" etc. but have not been successful to bypass it. I'll get back to you if I find a solution. Again, thanks so much for your wonderful work and getting back to me.

HI, I just encountered a similar issue, I've updated the os separator, but from the code I could only see that sugar_mesh_path is created, but the corresponding .ply is not, and then the code starts loading the mesh by o3d.io.read_triangle_mesh(sugar_mesh_path), which is impossible. So may I know how did you solved this issue?

Hi,

I'm kind of halfway through it. For now I managed to get the "coarse_mesh.py" script to create the :PLY file image

For it to create the .PLY file, in "coarse_mesh.py" I changed the following:

LINES 67-72: Change args.scene_path.split("/") FOR args.scene_path.split(os.sep)

Mesh output dir

 if args.mesh_output_dir is None:
        if len(args.scene_path.split(os.sep)[-1]) > 0: # if len(args.scene_path.split("/")[-1]) > 0:
            args.mesh_output_dir = os.path.join("./output/coarse_mesh", args.scene_path.split(os.sep)[-1]) #args.scene_path.split("/")[-1])
        else:
            args.mesh_output_dir = os.path.join("./output/coarse_mesh", args.scene_path.split(os.sep)[-2]) #args.scene_path.split("/")[-1])

LINE 477: Change sugarmesh_' + sugar_checkpoint_path.split('/') FOR sugarmesh_' + sugar_checkpoint_path.split(os.sep)

else:
      sugar_mesh_path = 'sugarmesh_' + sugar_checkpoint_path.split(os.sep)[-2].replace('sugarcoarse_', '') + '_levelZZ_decimAA.ply' #sugar_checkpoint_path.split('/')[-2].

LINE 610: Change sugarmesh_' + sugar_checkpoint_path.split('/') FOR sugarmesh_' + sugar_checkpoint_path.split(os.sep)

else:
  sugar_mesh_path = 'sugarmesh_' + sugar_checkpoint_path.split(os.sep)[-2].replace('sugarcoarse_', '') + '_poissoncenters_decimAA.ply' #sugar_checkpoint_path.split('/')

LINE 743: Change sugar_checkpoint_path.split('/') FOR sugar_checkpoint_path.split(os.sep)

else:
  sugar_mesh_path = 'sugarmesh_' + sugar_checkpoint_path.split(os.sep)[-2].replace('sugarcoarse_', '') + 'marchingcubes_levelZZ_decimAA.ply'

However, the "refined_mesh.py" halts when trying to open it cause it duplicates the path when trying to access it and can't find it (I don't know why it does it, but my guess is that I have to do the same "os.sep" changes for it to work but I haven't tried it yet). I'll try to sort this later, don't have the time right now, please let me know if you manage to fix it.

Best,

Nelson

citystrawman commented 10 months ago

Hi, Thank you for your quick response and great work. Your modeling and code writing is phenomenal. Yes, I'm trying to run it on windows. I saw a similar problem in another issue and I'm trying to fix it right now: I tried to change the .split("/") and "./output/coarse"; "/ouput/meshes" etc. but have not been successful to bypass it. I'll get back to you if I find a solution. Again, thanks so much for your wonderful work and getting back to me.

HI, I just encountered a similar issue, I've updated the os separator, but from the code I could only see that sugar_mesh_path is created, but the corresponding .ply is not, and then the code starts loading the mesh by o3d.io.read_triangle_mesh(sugar_mesh_path), which is impossible. So may I know how did you solved this issue?

Hi,

I'm kind of halfway through it. For now I managed to get the "coarse_mesh.py" script to create the :PLY file image

For it to create the .PLY file, in "coarse_mesh.py" I changed the following:

LINES 67-72: Change args.scene_path.split("/") FOR args.scene_path.split(os.sep)

Mesh output dir

if args.mesh_output_dir is None:
     if len(args.scene_path.split(os.sep)[-1]) > 0: # if len(args.scene_path.split("/")[-1]) > 0:
         args.mesh_output_dir = os.path.join("./output/coarse_mesh", args.scene_path.split(os.sep)[-1]) #args.scene_path.split("/")[-1])
     else:
         args.mesh_output_dir = os.path.join("./output/coarse_mesh", args.scene_path.split(os.sep)[-2]) #args.scene_path.split("/")[-1])

LINE 477: Change sugarmesh_' + sugar_checkpoint_path.split('/') FOR sugarmesh_' + sugar_checkpoint_path.split(os.sep)

else:
      sugar_mesh_path = 'sugarmesh_' + sugar_checkpoint_path.split(os.sep)[-2].replace('sugarcoarse_', '') + '_levelZZ_decimAA.ply' #sugar_checkpoint_path.split('/')[-2].

LINE 610: Change sugarmesh_' + sugar_checkpoint_path.split('/') FOR sugarmesh_' + sugar_checkpoint_path.split(os.sep)

else:
  sugar_mesh_path = 'sugarmesh_' + sugar_checkpoint_path.split(os.sep)[-2].replace('sugarcoarse_', '') + '_poissoncenters_decimAA.ply' #sugar_checkpoint_path.split('/')

LINE 743: Change sugar_checkpoint_path.split('/') FOR sugar_checkpoint_path.split(os.sep)

else:
  sugar_mesh_path = 'sugarmesh_' + sugar_checkpoint_path.split(os.sep)[-2].replace('sugarcoarse_', '') + 'marchingcubes_levelZZ_decimAA.ply'

However, the "refined_mesh.py" halts when trying to open it cause it duplicates the path when trying to access it and can't find it (I don't know why it does it, but my guess is that I have to do the same "os.sep" changes for it to work but I haven't tried it yet). I'll try to sort this later, don't have the time right now, please let me know if you manage to fix it.

Best,

Nelson

Hi, please refer to this post: https://github.com/Anttwo/SuGaR/issues/66 and I followed OuOu2021 and Kitmallet 's advices and then there's no issue about the directory inconsistency. but I still got one issue that sugar_mesh_path in refined_mesh.py is not correctly defined: sugar_mesh_path is the location of a .ply file that is saved in another place in previous stage. Thus I changed sugar_mesh_path as follows: image then I can run the whole program and get an obj mesh.
Hope you can make it :)

pyjacques commented 10 months ago

Hello, First thanks @Anttwo for this amazing work ! Making these adjustments to 'coarse_mesh.py' and 'refined_mesh.py', i've create my first refined Gaussians. 🤩 output files: image --> Refined .ply is OK, but no .obj ?! Command used: (under windows 11 microconda env (no linux)

cd conda\SuGaR
conda activate sugar
python train.py -s gaussian_splatting\input_data\h-pass-c-3e\ -c gaussian_splatting\input_data\h-pass-c-3e\output\ -r "density"

I've got an error at "Binding radiance cloud to surface mesh..." step :

...
Training finished after 15000 iterations with loss=0.022915204986929893.
Saving final model...
Final model saved.

Exporting ply file with refined Gaussians...
Ply file exported. This file is needed for using the dedicated viewer.
==================================================
Starting extracting texture from refined SuGaR model:
Scene path: gaussian_splatting\input_data\h-pass-c-3e\
Iteration to load: 7000
Vanilla 3DGS checkpoint path: gaussian_splatting\input_data\h-pass-c-3e\output\
Refined model path:
./output/refined\gaussian_splatting\input_data\h-pass-c-3e\sugarfine_coarse_mesh\h-pass-c-3e\h-pass-c-3e_level03_decim10
00000_normalconsistency01_gaussperface1/15000.pt
Coarse mesh path:
gaussian_splatting\input_data\h-pass-c-3e\gaussian_splatting\input_data\h-pass-c-3e\refined\gaussian_splatting\input_dat
a\h-pass-c-3e\sugarmesh_coarse_mesh\h-pass-c-3e\h-pass-c-3e_level03_decim1000000_normalconsistency01.ply
Mesh output directory: .\output\refined_mesh\gaussian_splatting\input_data\h-pass-c-3e\
Mesh save path:
.\output\refined_mesh\gaussian_splatting\input_data\h-pass-c-3e\refined\gaussian_splatting\input_data\h-pass-c-3e\sugarf
ine_coarse_mesh\h-pass-c-3e\h-pass-c-3e_level03_decim1000000_normalconsistency01_gaussperface1.obj
Number of gaussians per surface triangle: 1
Square size: 10
Postprocess mesh: False
==================================================
Source path: gaussian_splatting\input_data\h-pass-c-3e\
Gaussian splatting checkpoint path: gaussian_splatting\input_data\h-pass-c-3e\output\

Loading Vanilla 3DGS model config gaussian_splatting\input_data\h-pass-c-3e\output\...
Found image extension .png
Vanilla 3DGS Loaded.
65 training images detected.
The model has been trained for 7000 steps.
0.534775 M gaussians detected.
RPly: Unable to open file
[Open3D WARNING] Read PLY failed: unable to open file: gaussian_splatting\input_data\h-pass-c-3e\gaussian_splatting\input_data\h-pass-c-3e\refined\gaussian_splatting\input_data\h-pass-c-3e\sugarmesh_coarse_mesh\h-pass-c-3e\h-pass-c-3e_level03_decim1000000_normalconsistency01.ply
Binding radiance cloud to surface mesh...
Traceback (most recent call last):
  File "E:\conda\SuGaR\train.py", line 189, in <module>
    refined_mesh_path = extract_mesh_and_texture_from_refined_sugar(refined_mesh_args)
  File "E:\conda\SuGaR\sugar_extractors\refined_mesh.py", line 112, in extract_mesh_and_texture_from_refined_sugar
    refined_sugar = SuGaR(
  File "E:\conda\SuGaR\sugar_scene\sugar_model.py", line 323, in __init__
    scales = scales.clamp_min(0.0000001).reshape(len(faces_verts), -1, 1).expand(-1, self.n_gaussians_per_surface_triangle, 2).clone().reshape(-1, 2)
RuntimeError: cannot reshape tensor of 0 elements into shape [0, -1, 1] because the unspecified dimension size -1 can be any value and is ambiguous

(sugar) E:\conda\SuGaR>

In the other hand, I've tried also to setup conda env under ubuntu22 in WSL2/Win11 (cuda and tensorflow OK now)... I've found 2 guides : https://gist.github.com/neouranos/51dd12d9471eb73b2348d373b605b45a https://gist.github.com/Ayke/5f37ebdb84c758f57d7a3c8b847648bb

But i've got this error when i launch the script (repo cloned and untouched this time) :

(sugar) pyjacques@PC-PY-FIXE:~/SuGaR$ cd gaussian_splatting/submodules/diff-gaussian-rasterization/
pip install -e .
cd ../simple-knn/
pip install -e .
cd ../../../
Defaulting to user installation because normal site-packages is not writeable
Obtaining file:///home/pyjacques/SuGaR/gaussian_splatting/submodules/diff-gaussian-rasterization
  Preparing metadata (setup.py) ... done
Installing collected packages: diff-gaussian-rasterization
  Running setup.py develop for diff-gaussian-rasterization
Successfully installed diff-gaussian-rasterization-0.0.0
Defaulting to user installation because normal site-packages is not writeable
Obtaining file:///home/pyjacques/SuGaR/gaussian_splatting/submodules/simple-knn
  Preparing metadata (setup.py) ... done
Installing collected packages: simple-knn
  Running setup.py develop for simple-knn
Successfully installed simple-knn-0.0.0
(sugar) pyjacques@PC-PY-FIXE:~/SuGaR$ python train.py -s gaussian_splatting\input_data\h-pass-c-3e\ -c gaussian_splatting\input_data\h-pass-c-3e\output\ -r "density"
Traceback (most recent call last):
  File "/home/pyjacques/SuGaR/train.py", line 3, in <module>
    from sugar_trainers.coarse_density import coarse_training_with_density_regularization
  File "/home/pyjacques/SuGaR/sugar_trainers/coarse_density.py", line 7, in <module>
    from sugar_scene.gs_model import GaussianSplattingWrapper, fetchPly
  File "/home/pyjacques/SuGaR/sugar_scene/gs_model.py", line 6, in <module>
    from gaussian_splatting.scene.gaussian_model import GaussianModel
  File "/home/pyjacques/SuGaR/gaussian_splatting/scene/__init__.py", line 16, in <module>
    from scene.dataset_readers import sceneLoadTypeCallbacks
  File "/home/pyjacques/SuGaR/./gaussian_splatting/scene/__init__.py", line 16, in <module>
    from scene.dataset_readers import sceneLoadTypeCallbacks
  File "/home/pyjacques/SuGaR/./gaussian_splatting/scene/dataset_readers.py", line 24, in <module>
    from scene.gaussian_model import BasicPointCloud
  File "/home/pyjacques/SuGaR/./gaussian_splatting/scene/gaussian_model.py", line 20, in <module>
    from simple_knn._C import distCUDA2
ModuleNotFoundError: No module named 'simple_knn'
(sugar) pyjacques@PC-PY-FIXE:~/SuGaR$

I have "Successfully installed simple-knn-0.0.0" but "ModuleNotFoundError: No module named 'simple_knn'"

Trying again... Any advice?

This is my WSL2/Ubunto conda env :

(sugar) pyjacques@PC-PY-FIXE:~$ python3 collect_env.py
Collecting environment information...
PyTorch version: 2.0.1
Is debug build: False
CUDA used to build PyTorch: 11.8
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.3 LTS (x86_64)
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.9.18 (main, Sep 11 2023, 13:41:44)  [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.15.146.1-microsoft-standard-WSL2-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 11.8.89
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3090
Nvidia driver version: 551.23
cuDNN version: Probably one of the following:
/usr/local/cuda-11.8/targets/x86_64-linux/lib/libcudnn.so.8.6.0
/usr/local/cuda-11.8/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8.6.0
/usr/local/cuda-11.8/targets/x86_64-linux/lib/libcudnn_adv_train.so.8.6.0
/usr/local/cuda-11.8/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8.6.0
/usr/local/cuda-11.8/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8.6.0
/usr/local/cuda-11.8/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8.6.0
/usr/local/cuda-11.8/targets/x86_64-linux/lib/libcudnn_ops_train.so.8.6.0
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                       x86_64
CPU op-mode(s):                     32-bit, 64-bit
Address sizes:                      48 bits physical, 48 bits virtual
Byte Order:                         Little Endian
CPU(s):                             24
On-line CPU(s) list:                0-23
Vendor ID:                          AuthenticAMD
Model name:                         AMD Ryzen 9 5900X 12-Core Processor
CPU family:                         25
Model:                              33
Thread(s) per core:                 2
Core(s) per socket:                 12
Socket(s):                          1
Stepping:                           0
BogoMIPS:                           7400.08
Flags:                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core ssbd ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm
Virtualization:                     AMD-V
Hypervisor vendor:                  Microsoft
Virtualization type:                full
L1d cache:                          384 KiB (12 instances)
L1i cache:                          384 KiB (12 instances)
L2 cache:                           6 MiB (12 instances)
L3 cache:                           32 MiB (1 instance)
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit:        Not affected
Vulnerability L1tf:                 Not affected
Vulnerability Mds:                  Not affected
Vulnerability Meltdown:             Not affected
Vulnerability Mmio stale data:      Not affected
Vulnerability Retbleed:             Not affected
Vulnerability Spec rstack overflow: Mitigation; safe RET, no microcode
Vulnerability Spec store bypass:    Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:           Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:           Mitigation; Retpolines, IBPB conditional, IBRS_FW, STIBP conditional, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds:                Not affected
Vulnerability Tsx async abort:      Not affected

Versions of relevant libraries:
[pip3] numpy==1.26.2
[pip3] pytorch3d==0.7.4
[pip3] torch==2.0.1
[pip3] torchaudio==2.0.2
[pip3] torchvision==0.15.2
[pip3] triton==2.0.0
[conda] blas                      1.0                         mkl    conda-forge
[conda] ffmpeg                    4.3                  hf484d3e_0    pytorch
[conda] mkl                       2023.1.0         h213fc3f_46344
[conda] mkl-service               2.4.0            py39h5eee18b_1
[conda] mkl_fft                   1.3.8            py39h5eee18b_0
[conda] mkl_random                1.2.4            py39hdb19cb5_0
[conda] numpy                     1.26.2           py39h5f9d8c6_0
[conda] numpy-base                1.26.2           py39hb5e798b_0
[conda] pytorch                   2.0.1           py3.9_cuda11.8_cudnn8.7.0_0    pytorch
[conda] pytorch-cuda              11.8                 h7e8668a_5    pytorch
[conda] pytorch-mutex             1.0                        cuda    pytorch
[conda] pytorch3d                 0.7.4           py39_cu118_pyt201    pytorch3d
[conda] torchaudio                2.0.2                py39_cu118    pytorch
[conda] torchtriton               2.0.0                      py39    pytorch
[conda] torchvision               0.15.2               py39_cu118    pytorch

Hope you will have time to fix code for Windows soon 🤞