If my code extends your life, please say thanks to the GREAT authors of GS, then me. :-)

[yuedajiong]

rewrite, one night, just first version.
convergeable

done:

1. continue to optimize the code: correct, simple. <500 lines even <400 lines. (2023-10-21, corret!!!, mesh!!!) todo:
2. optimize algorithm. such as batchfiy, multi-object, pose-esitmation internal, priori/single-image(not dream-gauss); better viewer for this gauss data.
3. new algo, especial ‘Friendly’ to later mesh-extraction.

new code: (2023-11-15)

superi.zip

[goometasoft]

i can run offical code successfuly

but, when i run your code generate below error

iteration=000010  loss=0.000001
...
iteration=000600  loss=0.000004
Traceback (most recent call last):
  File "/mnt/d/Software/AI/gaus1/2310/wsl_cuda118/train2.py", line 456, in <module>
    main()

  File "/mnt/d/Software/AI/gaus1/2310/wsl_cuda118/train2.py", line 436, in main
    loss.backward()

  File "/opt/conda/envs/cuda118/lib/python3.9/site-packages/torch/_tensor.py", line 487, in backward
    torch.autograd.backward(

  File "/opt/conda/envs/cuda118/lib/python3.9/site-packages/torch/autograd/__init__.py", line 200, in backward
    Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
RuntimeError: Function _RasterizeGaussiansBackward returned an invalid gradient at index 2 - got [0, 0, 3] 
but expected shape compatible with [0, 16, 3]

[yuedajiong]

@1346052851

fixed a little bug: data-process, fore+back, *255. mesh is OK. tested, it is OK.
wizard.zip is my custom data.

[WWmore]

rewrite, one night, just first version. convergeable

done:

1. continue to optimize the code: correct, simple. <500 lines even <400 lines. (2023-10-21, corret!!!, mesh!!!) todo:
2. optimize algorithm. such as batchfiy, multi-object, pose-esitmation internal, priori/single-image(not dream-gauss); better viewer for this gauss data.
3. new algo, especial ‘Friendly’ to later mesh-extraction.

gs.py.txt ms.py.txt

wizard.zip

Thanks for your work. Could you please show how the output mesh of your wizard example look like? I run the code followd by convert.py --> train.py --> gs.py, which produces gs_shape.obj, gs_texture.obj and .mtl. But unfortunately the mesh looks quite messy and far from the wizard shape.

[520jz]

rewrite, one night, just first version. convergeable done:

1. continue to optimize the code: correct, simple. <500 lines even <400 lines. (2023-10-21, corret!!!, mesh!!!) todo:
2. optimize algorithm. such as batchfiy, multi-object, pose-esitmation internal, priori/single-image(not dream-gauss); better viewer for this gauss data.
3. new algo, especial ‘Friendly’ to later mesh-extraction.

gs.py.txt ms.py.txt wizard.zip

Thanks for your work. Could you please show how the output mesh of your wizard example look like? I run the code followd by convert.py --> train.py --> gs.py, which produces gs_shape.obj, gs_texture.obj and .mtl. But unfortunately the mesh looks quite messy and far from the wizard shape.

Hi! I have the same problem as you. Did you solve it？

[WWmore]

rewrite, one night, just first version. convergeable done:

1. continue to optimize the code: correct, simple. <500 lines even <400 lines. (2023-10-21, corret!!!, mesh!!!) todo:
2. optimize algorithm. such as batchfiy, multi-object, pose-esitmation internal, priori/single-image(not dream-gauss); better viewer for this gauss data.
3. new algo, especial ‘Friendly’ to later mesh-extraction.

gs.py.txt ms.py.txt wizard.zip

Thanks for your work. Could you please show how the output mesh of your wizard example look like? I run the code followd by convert.py --> train.py --> gs.py, which produces gs_shape.obj, gs_texture.obj and .mtl. But unfortunately the mesh looks quite messy and far from the wizard shape.

Hi! I have the same problem as you. Did you solve it？

Nope...

[yuedajiong]

@WWmore @520jz

hi, guys, there is not mesh-extraction in this official gaussian-splatting repo.
I used mesh-extraction code from DreamGaussian.

No any debug, no any optimization, so far. so, the quality of mesh is so so. :-(

in DreamGauss，it used zero123 priori for single image input. the output mesh is soso too (stage #1). but, the author designed some post-processing steps to optimize the quality.

I will do this optimization work, in later a few days. any progressing, tell you guys.

[520jz]

@WWmore @520jz

hi, guys, there is not mesh-extraction in this official gaussian-splatting repo. I used mesh-extraction code from DreamGaussian.

No any debug, no any optimization, so far. so, the quality of mesh is so so. :-(

I will do this optimization work, in later a few days.

any progressing, tell you guys.

Oh！Thank you very much！

[yuedajiong]

you can use this GS to generate ‘coarse’ mesh， then use other 'strong-fitting' algorithms to optimize: DreamGauss stage 2, DMTet, brute-force-fitting especial texuture in pytorch3d sampels, other interpolation algo, ...

in GS, if your GPU is not a problem, your can tune GS to samller Gauss points to get better result.

good luck.

[yuedajiong]

this gaussian-splatting is not born for surface/mesh.
not only representation, but also training.
the gauss point are not trend to converge to the surface of object.

the direct surface reconstuciton algorithms: SDF/UDF, NeuS/NeuS2/Neus2++, ...

[520jz]

you can use this GS to generate ‘coarse’ mesh， then use other 'strong-fitting' algorithms to optimize: DreamGauss stage 2, DMTet, brute-force-fitting especial texuture in pytorch3d sampels, other interpolation algo, ...

in GS, if your GPU is not a problem, your can tune GS to samller Gauss points to get better result.

good luck.

great answer！Thank you！

[HungNgoCT]

@yuedajiong ,

Mesh I obtained after running gs.py with wizard data like this. Not the object.

Any advice?

[cywm39]

i can run offical code successfuly

but, when i run your code generate below error

iteration=000010  loss=0.000001
...
iteration=000600  loss=0.000004
Traceback (most recent call last):
  File "/mnt/d/Software/AI/gaus1/2310/wsl_cuda118/train2.py", line 456, in <module>
    main()

  File "/mnt/d/Software/AI/gaus1/2310/wsl_cuda118/train2.py", line 436, in main
    loss.backward()

  File "/opt/conda/envs/cuda118/lib/python3.9/site-packages/torch/_tensor.py", line 487, in backward
    torch.autograd.backward(

  File "/opt/conda/envs/cuda118/lib/python3.9/site-packages/torch/autograd/__init__.py", line 200, in backward
    Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
RuntimeError: Function _RasterizeGaussiansBackward returned an invalid gradient at index 2 - got [0, 0, 3] 
but expected shape compatible with [0, 16, 3]

@yuedajiong

Hi! Thank you very much for uploading your code. But I met the same error:

RuntimeError: Function _RasterizeGaussiansBackward returned an invalid gradient at index 2 - got [0, 0, 3] but expected shape compatible with [0, 16, 3]

I notice that you have fixed the bug and uploaded gauss_rasterize.zip. But I don't know how to use it. There is no CMake file in it and even glm code in it is totally different. Could you tell me how to solve this problem?

[yuedajiong]

@cywm39 @1346052851

the code is OK to run, it is true that bug occasionally occur. I will debug and fix it. I tested and found: it looks if the scale is 'torch.ones((P, 3))', that means scale is 1,1,1, the bug will be trigered. Maybe not only my modified code. yes, this is a bug for all versions: https://github.com/graphdeco-inria/gaussian-splatting/issues/472

I am still woking on the mesh-extranction optimization. (there is not simple and 'GOOD' way to extract mesh from GS points) I added 'flat' logic to single gauss point, and 'flat' the gauss-points to close to surface. If you guys are interested in this trick, I can mail detials to you.

[yuedajiong]

@cywm39 you need not any Cmake operation.

these 2 lines will execute building automaticly.

you can turn on: verbose=True.

from torch.utils.cpp_extension import load  #~/.cache/torch_extensions/py310_cu117/GaussRasterize/
_C = load(name='GaussRasterize', sources=[gf+'/gauss_rasterize.cpp', gf+'/gauss_rasterize.cu', gf+'/cuda_rasterizer/forward.cu',gf+'/cuda_rasterizer/backward.cu',gf+'/cuda_rasterizer/rasterizer_impl.cu'], extra_include_paths=[gf+'/opengl_mathematics/'], extra_cflags=[''], verbose=False)

[juicee030]

rewrite, one night, just first version. convergeable

done:

1. continue to optimize the code: correct, simple. <500 lines even <400 lines. (2023-10-21, corret!!!, mesh!!!) todo:
2. optimize algorithm. such as batchfiy, multi-object, pose-esitmation internal, priori/single-image(not dream-gauss); better viewer for this gauss data.
3. new algo, especial ‘Friendly’ to later mesh-extraction.

new code: (2023-11-15)

superi.zip

I still can't get the correct mesh even with the new code :-(

[yuedajiong]

@juicee030 you can try other GS-mesh code firstly, like dream-gaussian (my mesh code is modified from it). I can train the GS and quality is OK, but the extracted mesh is NOT OK：not only texture, even pure shape. I checked the code, the extract_fields function can not extract correctly. I am still working on the problem, I will upload later code, extrac a better mesh, at least product-level.