the lego mesh result is not well

[chongxian]

hello, Thanks for your release code, I follow the Readme.md to do data preparation, download the lego.mp4 and run preprocessing command, then I run neuralangelo to train model , but until 100000 epoch the extract result is not well. could you kindly to give some advises.

[chenhsuanlin]

Hi @chongxian, could you verify if COLMAP found the correct camera poses?

[chongxian]

I load colmap data in blender, is it error?@chenhsuanlin

[chenhsuanlin]

You can use this notebook to help visualize the cameras and verify if the trajectory looks consistent with the images.

[altava-sgp]

@chenhsuanlin I got this mesh. ( resolution 512 )

This is normal ?

[mli0603]

Hi @altava-sgp

If this is from the lego dataset, this is not normal.

[altava-sgp]

@mli0603 What can I do ? This is my log.

Epoch: 499998, total time: 0.162679.                                                                                                                                                                               
Epoch: 499999, total time: 0.162837.                                                                                                                                                                               
Evaluating with 4 samples.                                                                                                                                                                                         
Training epoch 500000:   0%|                                                                                                                                                      | 0/1 [00:15<?, ?it/s, iter=5e+5]Done with training!!!
Saved checkpoint to logs/lego_group/lego_name/epoch_500000_iteration_000500000_checkpoint.pt                                                                                                                       
(neuralangelo) farer@farer-ubuntu-22:~/Documents/GitHub/neuralangelo$ 
(neuralangelo) farer@farer-ubuntu-22:~/Documents/GitHub/neuralangelo$ 
(neuralangelo) farer@farer-ubuntu-22:~/Documents/GitHub/neuralangelo$ torchrun --nproc_per_node=1 projects/neuralangelo/scripts/extract_mesh.py \
    --config=projects/neuralangelo/configs/custom/lego.yaml \
    --checkpoint=logs/lego_group/lego_name/epoch_500000_iteration_000500000_checkpoint.pt \
    --output_file=sample.ply \
    --resolution=2048 \
    --block_res=129
Running mesh extraction with 1 GPUs.
Setup trainer.
Using random seed 0
/home/farer/anaconda3/envs/neuralangelo/lib/python3.8/site-packages/tinycudann/modules.py:53: UserWarning: tinycudann was built for lower compute capability (86) than the system's (89). Performance may be suboptimal.
  warnings.warn(f"tinycudann was built for lower compute capability ({cc}) than the system's ({system_compute_capability}). Performance may be suboptimal.")
model parameter count: 366,702,732
Initialize model weights using type: none, gain: None
Using random seed 0
Allow TensorFloat32 operations on supported devices
Loading checkpoint (local): logs/lego_group/lego_name/epoch_500000_iteration_000500000_checkpoint.pt
- Loading the model...
Done with loading the checkpoint.
Extracting surface at resolution 2048 2048 2048
vertices: 28798162                                                                                                                                                                                                 
faces: 56691800
projects/neuralangelo/scripts/extract_mesh.py:101: DeprecationWarning: `remove_degenerate_faces` is deprecated and will be removed in March 2024 replace with `self.update_faces(self.nondegenerate_faces(height=height))`
  mesh.remove_degenerate_faces()
(neuralangelo) farer@farer-ubuntu-22:~/Documents/GitHub/neuralangelo$ 
(neuralangelo) farer@farer-ubuntu-22:~/Documents/GitHub/neuralangelo$ ll
total 1171924
drwxrwxr-x 11 farer farer       4096  8월 28 10:52 ./
drwxrwxr-x  7 farer farer       4096  8월 25 10:31 ../
drwxrwxr-x  2 farer farer       4096  8월 24 14:45 assets/
-rw-rw-r--  1 farer farer       6619  8월 24 14:45 DATA_PROCESSING.md
drwxrwxr-x  3 farer farer       4096  8월 24 17:53 datasets/
drwxrwxr-x  2 farer farer       4096  8월 24 14:45 docker/
drwxrwxr-x  9 farer farer       4096  8월 24 17:15 .git/
-rw-rw-r--  1 farer farer       3620  8월 24 14:45 .gitignore
-rw-rw-r--  1 farer farer        104  8월 24 14:45 .gitmodules
drwxrwxr-x  7 farer farer       4096  8월 24 16:48 imaginaire/
-rw-rw-r--  1 farer farer    2181280  8월 24 17:16 lego.mp4
-rw-rw-r--  1 farer farer       4454  8월 24 14:45 LICENSE.md
drwxrwxr-x  4 farer farer       4096  8월 24 17:59 logs/
-rw-rw-r--  1 farer farer        378  8월 24 14:45 neuralangelo.yaml
-rw-rw-r--  1 farer farer        143  8월 24 14:45 .pre-commit-config.yaml
drwxrwxr-x  4 farer farer       4096  8월 24 14:45 projects/
-rw-rw-r--  1 farer farer       6029  8월 24 14:45 README.md
-rw-rw-r--  1 farer farer        368  8월 24 14:45 requirements.txt
-rw-rw-r--  1 farer farer 1197759683  8월 28 10:52 sample.ply
drwxrwxr-x  3 farer farer       4096  8월 24 14:45 third_party/
drwxr-xr-x  4 farer farer       4096  8월 24 15:06 toy_example_skip24/
-rwxr-xr-x  1 farer farer        584  8월 24 15:06 toy_example.yaml*
-rw-rw-r--  1 farer farer       4130  8월 24 14:45 train.py
(neuralangelo) farer@farer-ubuntu-22:~/Documents/GitHub/neuralangelo$ 
(neuralangelo) farer@farer-ubuntu-22:~/Documents/GitHub/neuralangelo$ 
(neuralangelo) farer@farer-ubuntu-22:~/Documents/GitHub/neuralangelo$ torchrun --nproc_per_node=1 projects/neuralangelo/scripts/extract_mesh.py     --config=projects/neuralangelo/configs/custom/lego.yaml     --checkpoint=logs/lego_group/lego_name/epoch_500000_iteration_000500000_checkpoint.pt     --output_file=sample.ply     --resolution=512     --block_res=129 --textured
Running mesh extraction with 1 GPUs.
Setup trainer.
Using random seed 0
/home/farer/anaconda3/envs/neuralangelo/lib/python3.8/site-packages/tinycudann/modules.py:53: UserWarning: tinycudann was built for lower compute capability (86) than the system's (89). Performance may be suboptimal.
  warnings.warn(f"tinycudann was built for lower compute capability ({cc}) than the system's ({system_compute_capability}). Performance may be suboptimal.")
model parameter count: 366,702,732
Initialize model weights using type: none, gain: None
Using random seed 0
Allow TensorFloat32 operations on supported devices
Loading checkpoint (local): logs/lego_group/lego_name/epoch_500000_iteration_000500000_checkpoint.pt
- Loading the model...
Done with loading the checkpoint.
Extracting surface at resolution 512 512 512
vertices: 1793899                                                                                                                                                                                                  
faces: 3531860
colors: 1793899
[-0.8121788   0.0078125  -0.49609375] [-0.7734375  -0.21795682 -0.5859375 ] [-0.70126602 -0.109375   -0.51171875]
[163 125  37 255] [120 102  58 255] [203 146  31 255]
projects/neuralangelo/scripts/extract_mesh.py:101: DeprecationWarning: `remove_degenerate_faces` is deprecated and will be removed in March 2024 replace with `self.update_faces(self.nondegenerate_faces(height=height))`
  mesh.remove_degenerate_faces()
(neuralangelo) farer@farer-ubuntu-22:~/Documents/GitHub/neuralangelo$ 
(neuralangelo) farer@farer-ubuntu-22:~/Documents/GitHub/neuralangelo$ ll
total 1244796
drwxrwxr-x 11 farer farer       4096  8월 28 14:28 ./
drwxrwxr-x  7 farer farer       4096  8월 25 10:31 ../
drwxrwxr-x  2 farer farer       4096  8월 24 14:45 assets/
-rw-rw-r--  1 farer farer       6619  8월 24 14:45 DATA_PROCESSING.md
drwxrwxr-x  3 farer farer       4096  8월 24 17:53 datasets/
drwxrwxr-x  2 farer farer       4096  8월 24 14:45 docker/
drwxrwxr-x  9 farer farer       4096  8월 24 17:15 .git/
-rw-rw-r--  1 farer farer       3620  8월 24 14:45 .gitignore
-rw-rw-r--  1 farer farer        104  8월 24 14:45 .gitmodules
drwxrwxr-x  7 farer farer       4096  8월 24 16:48 imaginaire/
-rw-rw-r--  1 farer farer 1197759683  8월 28 10:52 lego_mesh.ply
-rw-rw-r--  1 farer farer    2181280  8월 24 17:16 lego.mp4
-rw-rw-r--  1 farer farer       4454  8월 24 14:45 LICENSE.md
drwxrwxr-x  4 farer farer       4096  8월 24 17:59 logs/
-rw-rw-r--  1 farer farer        378  8월 24 14:45 neuralangelo.yaml
-rw-rw-r--  1 farer farer        143  8월 24 14:45 .pre-commit-config.yaml
drwxrwxr-x  4 farer farer       4096  8월 24 14:45 projects/
-rw-rw-r--  1 farer farer       6029  8월 24 14:45 README.md
-rw-rw-r--  1 farer farer        368  8월 24 14:45 requirements.txt
-rw-rw-r--  1 farer farer   74616868  8월 28 14:28 sample.ply
drwxrwxr-x  3 farer farer       4096  8월 24 14:45 third_party/
drwxr-xr-x  4 farer farer       4096  8월 24 15:06 toy_example_skip24/
-rwxr-xr-x  1 farer farer        584  8월 24 15:06 toy_example.yaml*
-rw-rw-r--  1 farer farer       4130  8월 24 14:45 train.py
(neuralangelo) farer@farer-ubuntu-22:~/Documents/GitHub/neuralangelo$

[mli0603]

Hi @altava-sgp

Could you see if you can reproduce reasonable results with the colab?

[altava-sgp]

@mli0603 I will try when I am available.