search

ashawkey / nerf2mesh

[ICCV2023] Delicate Textured Mesh Recovery from NeRF via Adaptive Surface Refinement
https://me.kiui.moe/nerf2mesh/
MIT License
898 stars 88 forks source link

can not reach the proposed PSNR #90

Open garrisonz opened 1 year ago

garrisonz commented 1 year ago

Hi, Thanks for your great job. I reproduce nerf2mesh on my server (v100, 32G), no any code changed. But, for lego trial, I can ony reach 26.02 on stage0 and 23.78 on stage1, which is reported as 30.79, 29.67 in the paper.

The log about stage 0, and stage 1 shows following, any suggestion, please?

(pytorch3d) z@ubuntu:~/w/rsync/nerf2mesh/trial_syn_lego$ cat log_ngp_stage0.txt | grep -v '==>'
[INFO] Trainer: ngp_stage0 | 2023-09-12_18-08-30 | cuda | fp16 | trial_syn_lego/
[INFO] #parameters: 18367240
Namespace(path='data/nerf_synthetic/lego/', O=True, workspace='trial_syn_lego/', seed=0, stage=0, ckpt='latest', fp16=True, sdf=False, tcnn=False, progressive_level=False, test=False, test_no_video=False, test_no_mesh=False, camera_traj='', data_format='nerf', train_split='train', preload=True, random_image_batch=True, downscale=1, bound=1.0, scale=0.8, offset=[0, 0, 0], mesh='', enable_cam_near_far=False, enable_cam_center=False, min_near=0.05, enable_sparse_depth=False, enable_dense_depth=False, iters=30000, lr=0.01, lr_vert=0.0001, pos_gradient_boost=1, cuda_ray=True, max_steps=1024, update_extra_interval=16, max_ray_batch=4096, grid_size=128, mark_untrained=True, dt_gamma=0.0, density_thresh=10, diffuse_step=1000, diffuse_only=False, background='random', enable_offset_nerf_grad=False, n_eval=5, n_ckpt=50, num_rays=4096, adaptive_num_rays=True, num_points=262144, lambda_density=0, lambda_entropy=0, lambda_tv=1e-08, lambda_depth=0.1, lambda_specular=1e-05, lambda_eikonal=0.1, lambda_rgb=1, lambda_mask=0.1, wo_smooth=False, lambda_lpips=0, lambda_offsets=0.1, lambda_lap=0.001, lambda_normal=0, lambda_edgelen=0, contract=False, patch_size=1, trainable_density_grid=False, color_space='srgb', ind_dim=0, ind_num=500, mcubes_reso=512, env_reso=256, decimate_target=300000.0, mesh_visibility_culling=True, visibility_mask_dilation=5, clean_min_f=8, clean_min_d=5, ssaa=2, texture_size=4096, refine=True, refine_steps_ratio=[0.1, 0.2, 0.3, 0.4, 0.5, 0.7], refine_size=0.01, refine_decimate_ratio=0.1, refine_remesh_size=0.02, vis_pose=False, gui=False, W=1000, H=1000, radius=5, fovy=50, max_spp=1, refine_steps=[3000, 6000, 9000, 12000, 15000, 21000])
NeRFNetwork(
  (encoder): GridEncoder: input_dim=3 num_levels=16 level_dim=1 resolution=16 -> 2048 per_level_scale=1.3819 params=(6119864, 1) gridtype=hash align_corners=False interpolation=linear
  (sigma_net): MLP(
    (net): ModuleList(
      (0): Linear(in_features=19, out_features=32, bias=False)
      (1): Linear(in_features=32, out_features=1, bias=False)
    )
  )
  (encoder_color): GridEncoder: input_dim=3 num_levels=16 level_dim=2 resolution=16 -> 2048 per_level_scale=1.3819 params=(6119864, 2) gridtype=hash align_corners=False interpolation=linear
  (color_net): MLP(
    (net): ModuleList(
      (0): Linear(in_features=35, out_features=64, bias=False)
      (1): Linear(in_features=64, out_features=64, bias=False)
      (2): Linear(in_features=64, out_features=6, bias=False)
    )
  )
  (specular_net): MLP(
    (net): ModuleList(
      (0): Linear(in_features=6, out_features=32, bias=False)
      (1): Linear(in_features=32, out_features=3, bias=False)
    )
  )
)
[INFO] Loading latest checkpoint ...
[WARN] No checkpoint found, abort loading latest model.
++> Evaluate at epoch 545 ...
PSNR = 27.121713
++> Evaluate epoch 545 Finished, loss = 0.002292
[INFO] New best result: None --> 0.002292204032128211
++> Evaluate at epoch 1090 ...
PSNR = 27.356281
++> Evaluate epoch 1090 Finished, loss = 0.002181
[INFO] New best result: 0.002292204032128211 --> 0.0021811620131484233
++> Evaluate at epoch 1635 ...
PSNR = 27.395590
++> Evaluate epoch 1635 Finished, loss = 0.002169
[INFO] New best result: 0.0021811620131484233 --> 0.002168737300380599
++> Evaluate at epoch 2180 ...
PSNR = 27.376210
++> Evaluate epoch 2180 Finished, loss = 0.002188
++> Evaluate at epoch 2725 ...
PSNR = 27.321478
++> Evaluate epoch 2725 Finished, loss = 0.002231
[INFO] training takes 17.380065 minutes.
++> Evaluate at epoch 2728 ...
PSNR = 27.324507
SSIM = 0.917095
LPIPS (vgg) = 0.094983
++> Evaluate epoch 2728 Finished, loss = 0.002229
++> Evaluate at epoch 2728 ...
PSNR = 26.025525
SSIM = 0.907170
LPIPS (vgg) = 0.104211
++> Evaluate epoch 2728 Finished, loss = 0.002897
(pytorch3d) z@ubuntu:~/w/rsync/nerf2mesh/trial_syn_lego$ cat log_ngp_stage1.txt | grep -v '==>'
[INFO] Trainer: ngp_stage1 | 2023-09-12_19-34-12 | cuda | fp16 | trial_syn_lego/
[INFO] #parameters: 18824413
Namespace(path='data/nerf_synthetic/lego/', O=True, workspace='trial_syn_lego/', seed=0, stage=1, ckpt='latest', fp16=True, sdf=False, tcnn=False, progressive_level=False, test=False, test_no_video=False, test_no_mesh=False, camera_traj='', data_format='nerf', train_split='train', preload=True, random_image_batch=True, downscale=1, bound=1.0, scale=0.8, offset=[0, 0, 0], mesh='', enable_cam_near_far=False, enable_cam_center=False, min_near=0.05, enable_sparse_depth=False, enable_dense_depth=False, iters=30000, lr=0.01, lr_vert=0.0001, pos_gradient_boost=1, cuda_ray=True, max_steps=1024, update_extra_interval=16, max_ray_batch=4096, grid_size=128, mark_untrained=True, dt_gamma=0.0, density_thresh=10, diffuse_step=1000, diffuse_only=False, background='random', enable_offset_nerf_grad=False, n_eval=5, n_ckpt=50, num_rays=4096, adaptive_num_rays=True, num_points=262144, lambda_density=0, lambda_entropy=0, lambda_tv=1e-08, lambda_depth=0.1, lambda_specular=1e-05, lambda_eikonal=0.1, lambda_rgb=1, lambda_mask=0.1, wo_smooth=False, lambda_lpips=0, lambda_offsets=0.1, lambda_lap=0.001, lambda_normal=0, lambda_edgelen=0, contract=False, patch_size=1, trainable_density_grid=False, color_space='srgb', ind_dim=0, ind_num=500, mcubes_reso=512, env_reso=256, decimate_target=300000.0, mesh_visibility_culling=True, visibility_mask_dilation=5, clean_min_f=8, clean_min_d=5, ssaa=2, texture_size=4096, refine=True, refine_steps_ratio=[0.1, 0.2, 0.3, 0.4, 0.5, 0.7], refine_size=0.01, refine_decimate_ratio=0.1, refine_remesh_size=0.02, vis_pose=False, gui=False, W=1000, H=1000, radius=5, fovy=50, max_spp=1, refine_steps=[3000, 6000, 9000, 12000, 15000, 21000])
NeRFNetwork(
  (encoder): GridEncoder: input_dim=3 num_levels=16 level_dim=1 resolution=16 -> 2048 per_level_scale=1.3819 params=(6119864, 1) gridtype=hash align_corners=False interpolation=linear
  (sigma_net): MLP(
    (net): ModuleList(
      (0): Linear(in_features=19, out_features=32, bias=False)
      (1): Linear(in_features=32, out_features=1, bias=False)
    )
  )
  (encoder_color): GridEncoder: input_dim=3 num_levels=16 level_dim=2 resolution=16 -> 2048 per_level_scale=1.3819 params=(6119864, 2) gridtype=hash align_corners=False interpolation=linear
  (color_net): MLP(
    (net): ModuleList(
      (0): Linear(in_features=35, out_features=64, bias=False)
      (1): Linear(in_features=64, out_features=64, bias=False)
      (2): Linear(in_features=64, out_features=6, bias=False)
    )
  )
  (specular_net): MLP(
    (net): ModuleList(
      (0): Linear(in_features=6, out_features=32, bias=False)
      (1): Linear(in_features=32, out_features=3, bias=False)
    )
  )
)
[INFO] Loading stage 0 model to init stage 1 ...
[INFO] loaded model.
[WARN] missing keys: ['vertices_offsets']
[INFO] Loading latest checkpoint ...
[WARN] No checkpoint found, abort loading latest model.
[INFO] refine and decimate mesh at 3000 step
++> Evaluate at epoch 545 ...
PSNR = 25.259649
++> Evaluate epoch 545 Finished, loss = 0.003450
[INFO] New best result: None --> 0.003450471760588698
[INFO] refine and decimate mesh at 6000 step
[INFO] refine and decimate mesh at 9000 step
++> Evaluate at epoch 1090 ...
PSNR = 25.184187
++> Evaluate epoch 1090 Finished, loss = 0.003507
[INFO] refine and decimate mesh at 12000 step
[INFO] refine and decimate mesh at 15000 step
++> Evaluate at epoch 1635 ...
PSNR = 25.100105
++> Evaluate epoch 1635 Finished, loss = 0.003574
[INFO] refine and decimate mesh at 21000 step
++> Evaluate at epoch 2180 ...
PSNR = 24.992294
++> Evaluate epoch 2180 Finished, loss = 0.003657
++> Evaluate at epoch 2725 ...
PSNR = 24.841085
++> Evaluate epoch 2725 Finished, loss = 0.003774
[INFO] training takes 57.481197 minutes.
++> Evaluate at epoch 2728 ...
PSNR = 24.812209
SSIM = 0.874703
LPIPS (vgg) = 0.128298
++> Evaluate epoch 2728 Finished, loss = 0.003813
++> Evaluate at epoch 2728 ...
PSNR = 23.780770
SSIM = 0.864578
LPIPS (vgg) = 0.132985
++> Evaluate epoch 2728 Finished, loss = 0.004660