How to evaluate the Chamfer Distance of the DTU dataset?

[Ice-Tear]

Hi,thanks for your great work! I'm trying to implement exponentially decreasing eps and weight of curv loss (in Neuralangelo) in your code.It seems to work!

But I got a value of 7.6 when evaluating Chamfer Distance using the evaluate_single_scene.py in monosdf : (

I don't think the quality of this mesh should have such a high CD value, so I would like to ask you how to evaluate the Chamfer Distance of the DTU dataset?

[adkAurora]

Hi,thanks for your great work! I'm trying to implement exponentially decreasing eps and weight of curv loss (in Neuralangelo) in your code.It seems to work!

But I got a value of 7.6 when evaluating Chamfer Distance using the evaluate_single_scene.py in monosdf : (

I don't think the quality of this mesh should have such a high CD value, so I would like to ask you how to evaluate the Chamfer Distance of the DTU dataset?

could you please share the code？ My results on dtu_24 is poor, do you change the curv loss warmup step？

[bennyguo]

@Ice-Tear Could you check whether the extracted mesh and the ground truth mesh are in the same orientation? It's possible that you should rotate the extract mesh to align with the ground truth one.

[Ice-Tear]

@adkAurora yes,I change this file systems/neus.py .I add some code between self.log('train/loss_curvature',loss_curvature) and loss += loss_curvature * self.C(self.config.system.loss.lambda_curvature). Here is the code.

            # warm up the weight of curv loss
            if self.global_step < self.config.system.loss.curvature_warmup_steps:
                curvature_weight = (self.global_step+1) * self.C(self.config.system.loss.lambda_curvature) / self.config.system.loss.curvature_warmup_steps
            else:
                # decay the weight of curv loss
                growth_ratio = self.config.model.geometry.xyz_encoding_config.per_level_scale
                growth_steps = self.config.model.geometry.xyz_encoding_config.update_steps
                curvature_weight = np.exp(-max(self.global_step - self.config.system.loss.curvature_warmup_steps,0) * np.log(growth_ratio) / growth_steps) * self.C(self.config.system.loss.lambda_curvature)

And I change the config file.

[Ice-Tear]

@bennyguo Thank you for your reply! I just started learning 3D reconstruction and I don't know how to check orientation yet.But when I open them with meshlab, their orientation looks the same.How do you usually evaluate the Chamfer Distance of DTU?

[yizhidecainiao]

Hi,thanks for your great work! I'm trying to implement exponentially decreasing eps and weight of curv loss (in Neuralangelo) in your code.It seems to work!

But I got a value of 7.6 when evaluating Chamfer Distance using the evaluate_single_scene.py in monosdf : (

I don't think the quality of this mesh should have such a high CD value, so I would like to ask you how to evaluate the Chamfer Distance of the DTU dataset?

Thank you for your code, can you provide your configuration file, I tried it myself and did not get as good results as you did.

[Ice-Tear]

@yizhidecainiao Here is the config. name: neuralangelo-dtu-wmask-${basename:${dataset.root_dir}} tag: "" seed: 42

dataset: name: dtu root_dir: ./load/DTU-neus/dtu_scan24 cameras_file: cameras_sphere.npz img_downscale: 2 # specify training image size by either img_wh or img_downscale n_test_traj_steps: 60 apply_mask: true

model: name: neus radius: 1.0 num_samples_per_ray: 1024 train_num_rays: 512 max_train_num_rays: 2048 grid_prune: false grid_prune_occ_thre: 0.001 dynamic_ray_sampling: true batch_image_sampling: true randomized: true ray_chunk: 2048 cos_anneal_end: 500000 learned_background: false background_color: white variance: init_val: 0.3 modulate: false geometry: name: volume-sdf radius: 1.0 feature_dim: 13 grad_type: finite_difference finite_difference_eps: progressive isosurface: method: mc resolution: 512 chunk: 2097152 threshold: 0. xyz_encoding_config: otype: ProgressiveBandHashGrid n_levels: 16 n_features_per_level: 8 log2_hashmap_size: 22 base_resolution: 32 per_level_scale: 1.3195079565048218 include_xyz: true start_level: 4 start_step: 0 update_steps: 5000 finest_size_ratio: 1. mlp_network_config: otype: VanillaMLP activation: ReLU output_activation: none n_neurons: 64 n_hidden_layers: 1 sphere_init: true sphere_init_radius: 0.5 weight_norm: true texture: name: volume-radiance input_feature_dim: ${add:${model.geometry.feature_dim},3} # surface normal as additional input dir_encoding_config: otype: SphericalHarmonics degree: 4 mlp_network_config: otype: VanillaMLP activation: ReLU output_activation: none n_neurons: 64 n_hidden_layers: 4 color_activation: sigmoid

system: name: neus-system loss: lambda_rgb_mse: 0. lambda_rgb_l1: 1. lambda_mask: 0.1 lambda_eikonal: 0.1 lambda_curvature: 0.0005 curvature_warmup_steps: 5000 lambda_sparsity: 0.0 lambda_distortion: 0.0 lambda_distortion_bg: 0.0 lambda_opaque: 0.0 sparsity_scale: 1. optimizer: name: AdamW args: lr: 0.01 betas:

• 0.9
• 0.99 eps: 1.e-15 params: geometry: lr: 0.01 texture: lr: 0.01 variance: lr: 0.001 constant_steps: 5000 scheduler: name: SequentialLR interval: step milestones:
  • 5000 schedulers:
  • name: ConstantLR args: factor: 1.0 total_iters: 5000
  • name: ExponentialLR args: gamma: 0.9999953483237626

checkpoint: save_top_k: -1 every_n_train_steps: 250000

export: chunk_size: 2097152 export_vertex_color: True

trainer: max_steps: 500000 log_every_n_steps: 100 num_sanity_val_steps: 0 val_check_interval: 5000 limit_train_batches: 1.0 limit_val_batches: 2 enable_progress_bar: true precision: 16

[yizhidecainiao]

@yizhidecainiao Here is the config. name: neuralangelo-dtu-wmask-${basename:${dataset.root_dir}} tag: "" seed: 42

dataset: name: dtu root_dir: ./load/DTU-neus/dtu_scan24 cameras_file: cameras_sphere.npz img_downscale: 2 # specify training image size by either img_wh or img_downscale n_test_traj_steps: 60 apply_mask: true

model: name: neus radius: 1.0 num_samples_per_ray: 1024 train_num_rays: 512 max_train_num_rays: 2048 grid_prune: false grid_prune_occ_thre: 0.001 dynamic_ray_sampling: true batch_image_sampling: true randomized: true ray_chunk: 2048 cos_anneal_end: 500000 learned_background: false background_color: white variance: init_val: 0.3 modulate: false geometry: name: volume-sdf radius: 1.0 feature_dim: 13 grad_type: finite_difference finite_difference_eps: progressive isosurface: method: mc resolution: 512 chunk: 2097152 threshold: 0. xyz_encoding_config: otype: ProgressiveBandHashGrid n_levels: 16 n_features_per_level: 8 log2_hashmap_size: 22 base_resolution: 32 per_level_scale: 1.3195079565048218 include_xyz: true start_level: 4 start_step: 0 update_steps: 5000 finest_size_ratio: 1. mlp_network_config: otype: VanillaMLP activation: ReLU output_activation: none n_neurons: 64 n_hidden_layers: 1 sphere_init: true sphere_init_radius: 0.5 weight_norm: true texture: name: volume-radiance input_feature_dim: ${add:${model.geometry.feature_dim},3} # surface normal as additional input dir_encoding_config: otype: SphericalHarmonics degree: 4 mlp_network_config: otype: VanillaMLP activation: ReLU output_activation: none n_neurons: 64 n_hidden_layers: 4 color_activation: sigmoid

system: name: neus-system loss: lambda_rgb_mse: 0. lambda_rgb_l1: 1. lambda_mask: 0.1 lambda_eikonal: 0.1 lambda_curvature: 0.0005 curvature_warmup_steps: 5000 lambda_sparsity: 0.0 lambda_distortion: 0.0 lambda_distortion_bg: 0.0 lambda_opaque: 0.0 sparsity_scale: 1. optimizer: name: AdamW args: lr: 0.01 betas: - 0.9 - 0.99 eps: 1.e-15 params: geometry: lr: 0.01 texture: lr: 0.01 variance: lr: 0.001 constant_steps: 5000 scheduler: name: SequentialLR interval: step milestones: - 5000 schedulers: - name: ConstantLR args: factor: 1.0 total_iters: 5000 - name: ExponentialLR args: gamma: 0.9999953483237626

checkpoint: save_top_k: -1 every_n_train_steps: 250000

export: chunk_size: 2097152 export_vertex_color: True

trainer: max_steps: 500000 log_every_n_steps: 100 num_sanity_val_steps: 0 val_check_interval: 5000 limit_train_batches: 1.0 limit_val_batches: 2 enable_progress_bar: true precision: 16

Thanks for your reply!

[ccetaw]

@bennyguo Thank you for your reply! I just started learning 3D reconstruction and I don't know how to check orientation yet.But when I open them with meshlab, their orientation looks the same.How do you usually evaluate the Chamfer Distance of DTU?

Hi! I also encountered the same problem. Did you find the correct orientation?