NVlabs / neuralangelo

Official implementation of "Neuralangelo: High-Fidelity Neural Surface Reconstruction" (CVPR 2023)
https://research.nvidia.com/labs/dir/neuralangelo/
Other
4.33k stars 387 forks source link

Still no texture after add the --textured #62

Closed dolcelee closed 1 year ago

dolcelee commented 1 year ago

First of all, amazing work! I tried a portrait dataset, the mesh looks good. However, it still shows no texture after I added "--textured". I use Meshlab and Blender to view ply files. Did I miss anything? 微信截图_20230823102757 CHECKPOINT=logs/example_group/example_name/xxx.pt OUTPUT_MESH=xxx.ply CONFIG=projects/neuralangelo/configs/custom/${EXPERIMENT}.yaml RESOLUTION=2048 BLOCK_RES=128 GPUS=1 # use >1 for multi-GPU mesh extraction torchrun --nproc_per_node=${GPUS} projects/neuralangelo/scripts/extract_mesh.py \ --config=${CONFIG} \ --checkpoint=${CHECKPOINT} \ --output_file=${OUTPUT_MESH} \ --resolution=${RESOLUTION} \ --block_res=${BLOCK_RES} \ --textured

chenhsuanlin commented 1 year ago

Thanks for reporting. This is a known bug, will fix it in the coming days.

ftn-gautam commented 1 year ago

I see the same issue but for me, it shows texture if I use the first checkpoint but no texture for checkpoints after the first one.

iam-machine commented 1 year ago

@chenhsuanlin Is it really a bug? It didn't work for me when I wrote it the way the author of the issue did, but when I wrote it the different way everything worked. Just placed the command in the beginning and added backlash after it.

--textured \ --config=${CONFIG} \ --checkpoint=${CHECKPOINT} \ --output_file=${OUTPUT_MESH} \ --resolution=${RESOLUTION} \ --block_res=${BLOCK_RES}

Bai426 commented 1 year ago

I have the same issue.

dolcelee commented 1 year ago

@chenhsuanlin Is it really a bug? It didn't work for me when I wrote it the way the author of the issue did, but when I wrote it the different way everything worked. Just placed the command in the beginning and added backlash after it.

--textured --config=${CONFIG} --checkpoint=${CHECKPOINT} --output_file=${OUTPUT_MESH} --resolution=${RESOLUTION} --block_res=${BLOCK_RES}

Thanks for the data point, I tried this way but still didn't work. However, when I changed to the first checkpoint instead of others, the texture shows.

dolcelee commented 1 year ago

I see the same issue but for me, it shows texture if I use the first checkpoint but no texture for checkpoints after the first one. 微信图片_20230824102043

Thanks for the data point, I tried the first checkpoint, the texture shows as you said! The performance is not good unfortunately

Bai426 commented 1 year ago

@dolcelee Sorry, I didn't see the texture yet with the first checkpoint and the above scripts, not mentioned using other checkpoints. I also noticed that the 3d model is very rough in the first checkpoint.

mli0603 commented 1 year ago

Hi @dolcelee @Bai426 @ftn-gautam

Sorry for the late reply.

dolcelee commented 1 year ago

@mli0603

mli0603 commented 1 year ago

Thank you @dolcelee

This is likely a bug related to #75 where the loaded checkpoints fail to predict any colors. We are looking into this now and will circle back.

mli0603 commented 1 year ago

We have pushed a commit that potentially fixes the issue of mesh extraction (https://github.com/NVlabs/neuralangelo/commit/c91af8d5098c858df8e8dfa35fba8666d314782b). Please let us know if you still run into the same problem.

Bai426 commented 1 year ago

Thank you very much.Yes, I tested and found that the texture can be seen on any checkpoint. The only issue is that the reconstruction result has some noises like what dolcelee showed above.

Best, Mike

dolcelee commented 1 year ago

@mli0603 I retrained the same dataset and regenerated the mesh. It shows the texture but looks weird. I think this problem is related to the bounding sphere. Could you please tell me how to set the value of the sphere? _tmp_dingtalkgov_qt_pic_1693444522195

mli0603 commented 1 year ago

Hi @dolcelee

Good to know that textured mesh can be extracted. For sphere adjustment, please see DATA PREPROCESSING.

mli0603 commented 1 year ago

Closing this issue as it is resolved. For other issues, feel free to report in a separate thread.

lie12huo commented 1 year ago

@mli0603 I used the latest version of the code, but the exported mesh still has no texture. Is there anything else I need to pay attention to?My export script is as follows:

_GROUP=fc_example NAME=fc_example_0831 CHECKPOINT=logs/${GROUP}/${NAME}/epoch_00053_iteration_000020000_checkpoint.pt OUTPUT_MESH=yzfc_53.ply CONFIG=logs/${GROUP}/${NAME}/config.yaml RESOLUTION=2048 BLOCK_RES=128 GPUS=1 # use >1 for multi-GPU mesh extraction torchrun --nproc_per_node=${GPUS} projects/neuralangelo/scripts/extract_mesh.py \ --textured \ --config=${CONFIG} \ --checkpoint=${CHECKPOINT} \ --output_file=${OUTPUT_MESH} \ --resolution=${RESOLUTION} \ --block_res=${BLOCKRES} ad060d8b14109908a918917cdec8aea

chenhsuanlin commented 1 year ago

@lie12huo the checkpoint is likely corrupted from the previous (buggy) code. You would need to retrain the model for this case, not just re-extract the mesh.

lie12huo commented 1 year ago

@chenhsuanlin I used the latest code to preprocess and train checkpoints today, but the exported model still has no texture. image