Parameters for the TNT-dataset

ZirongChan commented 11 months ago

Hello, I've trained the meetingroom from TNT, using batch_size = 16, for 500000 iterations, with the bounding sphere scale = 1. the other parameters are set as default settings given in the base.yaml and the colmap outputs, the hashgrid dict_size is set to 22, and I also use the apper_embed option since I found out that you use it in the supplementary material.

The result looks good, but still, it is not as good as the one you demonstrated, mine lacks of details and sharpness. what I've got: what you show:

I wonder what parameters else can I adjust to get a better result? should I train it for more iterations such as 1000K or set the bounding sphere more thigh to the region which contains the valid sparse points from colmap?

ZirongChan commented 11 months ago

this is what I get after training for 100K iterations with the same parameters I mentioned.

I think that the number of the iterations does not help much, I wonder what goes wrong in my setting.

mli0603 commented 11 months ago

Hi @ZirongChan

Yes, your result indeed is missing the details. Can you provide the following?

The commit you are on
A screenshot of the training metrics (both losses and variables monitored)

ZirongChan commented 11 months ago

Hi @ZirongChan

Yes, your result indeed is missing the details. Can you provide the following?
* The commit you are on

* A screenshot of the training metrics (both losses and variables monitored)

@mli0603 thx for your reply. 1, I'm pretty sure that the commit is "f512b278" 2, I did not manage to get the code running with wandb since some unknown issue, so I don't have the training metrics monitoring results. Is there something else that I can provide that is useful for analyzing the problem?

mli0603 commented 11 months ago

Could you look into wandb logging? It will be beneficial to have them for debugging purposes. I wonder if something is wrong with the setup and some metrics have gone wild.

qq297110281 commented 11 months ago

i try meetingroom too, the result look not good. after 500k iterations: snapshot02 snapshot01 snapshot00

i am using RTX 2080 Ti with 8G memory here is the parameters: Training with 1 GPUs. Using random seed 0 Make folder logs/meetingroom

checkpoint:
- save_epoch: 9999999999
- save_iter: 20000
- save_latest_iter: 9999999999
- save_period: 9999999999
- strict_resume: True
cudnn:
- benchmark: True
- deterministic: False
data:
- name: dummy
- num_images: 371
- num_workers: 4
- preload: True
- readjust:
  - center: [0.0, 0.0, 0.0]
  - scale: 1.0
- root: datasets/tanks_and_temples/Meetingroom
- train:
  - batch_size: 1
  - image_size: [835, 1500]
  - subset: None
- type: projects.neuralangelo.data
- use_multi_epoch_loader: True
- val:
  - batch_size: 1
  - image_size: [300, 540]
  - max_viz_samples: 16
  - subset: 1
image_save_iter: 9999999999
inference_args:
local_rank: 0
logdir: logs/meetingroom
logging_iter: 9999999999999
max_epoch: 9999999999
max_iter: 500000
metrics_epoch: None
metrics_iter: None
model:
- appear_embed:
  - dim: 8
  - enabled: True
- background:
  - enabled: True
  - encoding:
    - levels: 10
    - type: fourier
  - encoding_view:
    - levels: 3
    - type: spherical
  - mlp:
    - activ: relu
    - activ_density: softplus
    - activ_density_params:
    - activ_params:
    - hidden_dim: 256
    - hidden_dim_rgb: 128
    - num_layers: 8
    - num_layers_rgb: 2
    - skip: [4]
    - skip_rgb: []
  - view_dep: True
  - white: False
- object:
  - rgb:
    - encoding_view:
    - levels: 3
    - type: spherical
    - mlp:
    - activ: relu_
    - activ_params:
    - hidden_dim: 256
    - num_layers: 4
    - skip: []
    - weight_norm: True
    - mode: idr
  - s_var:
    - anneal_end: 0.1
    - init_val: 3.0
  - sdf:
    - encoding:
    - coarse2fine:
      - enabled: True
      - init_active_level: 8
      - step: 5000
    - hashgrid:
      - dict_size: 20
      - dim: 8
      - max_logres: 11
      - min_logres: 5
      - range: [-2, 2]
    - levels: 16
    - type: hashgrid
    - gradient:
    - mode: numerical
    - taps: 4
    - mlp:
    - activ: softplus
    - activ_params:
      - beta: 100
    - geometric_init: True
    - hidden_dim: 256
    - inside_out: True
    - num_layers: 1
    - out_bias: 0.5
    - skip: []
    - weight_norm: True
- render:
  - num_sample_hierarchy: 4
  - num_samples:
    - background: 32
    - coarse: 64
    - fine: 16
  - rand_rays: 512
  - stratified: True
- type: projects.neuralangelo.model
nvtx_profile: False
optim:
- fused_opt: False
- params:
  - lr: 0.001
  - weight_decay: 0.01
- sched:
  - gamma: 10.0
  - iteration_mode: True
  - step_size: 9999999999
  - two_steps: [300000, 400000]
  - type: two_steps_with_warmup
  - warm_up_end: 5000
- type: AdamW
pretrained_weight: None
source_filename: projects/neuralangelo/configs/tnt.yaml
speed_benchmark: False
test_data:
- name: dummy
- num_workers: 0
- test:
  - batch_size: 1
  - is_lmdb: False
  - roots: None
- type: imaginaire.datasets.images
timeout_period: 9999999
trainer:
- amp_config:
  - backoff_factor: 0.5
  - enabled: False
  - growth_factor: 2.0
  - growth_interval: 2000
  - init_scale: 65536.0
- ddp_config:
  - find_unused_parameters: False
  - static_graph: True
- depth_vis_scale: 0.5
- ema_config:
  - beta: 0.9999
  - enabled: False
  - load_ema_checkpoint: False
  - start_iteration: 0
- grad_accum_iter: 1
- image_to_tensorboard: False
- init:
  - gain: None
  - type: none
- loss_weight:
  - curvature: 0.0005
  - eikonal: 0.1
  - render: 1.0
- type: projects.neuralangelo.trainer
validation_iter: 5000
wandb_image_iter: 10000
wandb_scalar_iter: 100 cudnn benchmark: True cudnn deterministic: False Setup trainer. Using random seed 0 model parameter count: 106,268,540 Initialize model weights using type: none, gain: None Using random seed 0 Allow TensorFloat32 operations on supported devices

and the config file : 1、projects/neuralangelo/configs/tnt.yaml

parent: projects/neuralangelo/configs/base.yaml

model: object: sdf: mlp: inside_out: True # True for Meetingroom. encoding: coarse2fine: init_active_level: 8 appear_embed: enabled: True dim: 8

data: type: projects.neuralangelo.data root: datasets/tanks_and_temples/Meetingroom num_images: 371 # The number of training images. train: image_size: [835,1500] batch_size: 1 subset: val: image_size: [300,540] batch_size: 1 subset: 1 max_viz_samples: 16

2、projects/neuralangelo/configs/base.yaml logging_iter: 9999999999999 # disable the printing logger

max_iter: 500000

wandb_scalar_iter: 100 wandb_image_iter: 10000 validation_iter: 5000 speed_benchmark: False

checkpoint: save_iter: 20000

trainer: type: projects.neuralangelo.trainer ema_config: enabled: False load_ema_checkpoint: False loss_weight: render: 1.0 eikonal: 0.1 curvature: 5e-4 init: type: none amp_config: enabled: False depth_vis_scale: 0.5

model: type: projects.neuralangelo.model object: sdf: mlp: num_layers: 1 hidden_dim: 256 skip: [] activ: softplus activ_params: beta: 100 geometric_init: True weight_norm: True out_bias: 0.5 inside_out: False encoding: type: hashgrid levels: 16 hashgrid: min_logres: 5 max_logres: 11 dict_size: 20 dim: 8 range: [-2,2] coarse2fine: enabled: True init_active_level: 4 step: 5000 gradient: mode: numerical taps: 4 rgb: mlp: num_layers: 4 hiddendim: 256 skip: [] activ: relu activ_params: {} weight_norm: True mode: idr encoding_view: type: spherical levels: 3 s_var: init_val: 3. anneal_end: 0.1 background: enabled: True white: False mlp: num_layers: 8 hidden_dim: 256 skip: [4] num_layers_rgb: 2 hidden_dim_rgb: 128 skip_rgb: [] activ: relu activ_params: {} activ_density: softplus activ_density_params: {} view_dep: True encoding: type: fourier levels: 10 encoding_view: type: spherical levels: 3 render: rand_rays: 512 num_samples: coarse: 64 fine: 16 background: 32 num_sample_hierarchy: 4 stratified: True appear_embed: enabled: False dim: 8

optim: type: AdamW params: lr: 1e-3 weight_decay: 1e-2 sched: iteration_mode: True type: two_steps_with_warmup warm_up_end: 5000 two_steps: [300000,400000] gamma: 10.0

data: type: projects.nerf.datasets.nerf_blender root: datasets/nerf-synthetic/lego use_multi_epoch_loader: True num_workers: 4 preload: True num_images: # The number of training images. train: image_size: [800,800] batch_size: 2 subset: val: image_size: [400,400] batch_size: 2 subset: 4 max_viz_samples: 16 readjust: center: [0.,0.,0.] scale: 1.

by the way ，I found only scene_type=object which video shoot for object can get good result.

mli0603 commented 11 months ago

Hi @qq297110281

We scale the compute up by using more GPUs. We find it is important to scale the batch size up for better results (such as 16 for TnT). This is detailed in the appenidx.

qq297110281 commented 10 months ago

@mli0603 you mean like this ?

Training with 1 GPUs. Using random seed 0 Make folder logs/meetingroom

checkpoint:
- save_epoch: 9999999999
- save_iter: 20000
- save_latest_iter: 9999999999
- save_period: 9999999999
- strict_resume: True
cudnn:
- benchmark: True
- deterministic: False
data:
- name: dummy
- num_images: 371
- num_workers: 4
- preload: True
- readjust:
  - center: [0.0, 0.0, 0.0]
  - scale: 1.0
- root: datasets/tanks_and_temples/Meetingroom
- train:
  - batch_size: 16
  - image_size: [835, 1500]
  - subset: None
- type: projects.neuralangelo.data
- use_multi_epoch_loader: True
- val:
  - batch_size: 16
  - image_size: [300, 540]
  - max_viz_samples: 16
  - subset: 1
image_save_iter: 9999999999
inference_args:
local_rank: 0
logdir: logs/meetingroom
logging_iter: 9999999999999
max_epoch: 9999999999
max_iter: 500000
metrics_epoch: None
metrics_iter: None
model:
- appear_embed:
  - dim: 8
  - enabled: True
- background:
  - enabled: True
  - encoding:
    - levels: 10
    - type: fourier
  - encoding_view:
    - levels: 3
    - type: spherical
  - mlp:
    - activ: relu
    - activ_density: softplus
    - activ_density_params:
    - activ_params:
    - hidden_dim: 256
    - hidden_dim_rgb: 128
    - num_layers: 8
    - num_layers_rgb: 2
    - skip: [4]
    - skip_rgb: []
  - view_dep: True
  - white: False
- object:
  - rgb:
    - encoding_view:
    - levels: 3
    - type: spherical
    - mlp:
    - activ: relu_
    - activ_params:
    - hidden_dim: 256
    - num_layers: 4
    - skip: []
    - weight_norm: True
    - mode: idr
  - s_var:
    - anneal_end: 0.1
    - init_val: 3.0
  - sdf:
    - encoding:
    - coarse2fine:
      - enabled: True
      - init_active_level: 8
      - step: 5000
    - hashgrid:
      - dict_size: 22
      - dim: 8
      - max_logres: 11
      - min_logres: 5
      - range: [-2, 2]
    - levels: 16
    - type: hashgrid
    - gradient:
    - mode: numerical
    - taps: 4
    - mlp:
    - activ: softplus
    - activ_params:
      - beta: 100
    - geometric_init: True
    - hidden_dim: 256
    - inside_out: True
    - num_layers: 1
    - out_bias: 0.5
    - skip: []
    - weight_norm: True
- render:
  - num_sample_hierarchy: 4
  - num_samples:
    - background: 32
    - coarse: 64
    - fine: 16
  - rand_rays: 512
  - stratified: True
- type: projects.neuralangelo.model
nvtx_profile: False
optim:
- fused_opt: False
- params:
  - lr: 0.001
  - weight_decay: 0.01
- sched:
  - gamma: 10.0
  - iteration_mode: True
  - step_size: 9999999999
  - two_steps: [300000, 400000]
  - type: two_steps_with_warmup
  - warm_up_end: 5000
- type: AdamW
pretrained_weight: None
source_filename: projects/neuralangelo/configs/tnt.yaml
speed_benchmark: False
test_data:
- name: dummy
- num_workers: 0
- test:
  - batch_size: 1
  - is_lmdb: False
  - roots: None
- type: imaginaire.datasets.images
timeout_period: 9999999
trainer:
- amp_config:
  - backoff_factor: 0.5
  - enabled: False
  - growth_factor: 2.0
  - growth_interval: 2000
  - init_scale: 65536.0
- ddp_config:
  - find_unused_parameters: False
  - static_graph: True
- depth_vis_scale: 0.5
- ema_config:
  - beta: 0.9999
  - enabled: False
  - load_ema_checkpoint: False
  - start_iteration: 0
- grad_accum_iter: 1
- image_to_tensorboard: False
- init:
  - gain: None
  - type: none
- loss_weight:
  - curvature: 0.0005
  - eikonal: 0.1
  - render: 1.0
- type: projects.neuralangelo.trainer
validation_iter: 5000
wandb_image_iter: 10000
wandb_scalar_iter: 100 cudnn benchmark: True cudnn deterministic: False Setup trainer. Using random seed 0 model parameter count: 366,711,740 Initialize model weights using type: none, gain: None Using random seed 0 Allow TensorFloat32 operations on supported devices Train dataset length: 49
Val dataset length: 1

in the paper, for tnt dataset, the marching cubes resolution is 2048, is that mean extract mesh using --resolution=2048 ?

i try the config above, using 1 A100 80G GPU, but got strange result after 100k iterations:

extract mesh using: torchrun --nproc_per_node=1 projects/neuralangelo/scripts/extract_mesh.py --config=logs/meetingroom/config.yaml --checkpoint=logs/meetingroom/epoch_33332_iteration_000100000_checkpoint.pt --output_file=/home/nlt/meetingroom_2k_tx.ply --resolution=2048 --block_res=128 --textured

NVlabs / neuralangelo

Parameters for the TNT-dataset #113