Regular noises during Zero-1-to-3 training

[freemantom]

When I try to train Zero-1-to-3 with the default config but longer training steps: python launch.py --config configs/zero123.yaml --train --gpu 0 data.image_path=./load/images/dog1_rgba.png trainer and changes the following configs in configs/zero123.yaml: `trainer: max_steps: 800 log_every_n_steps: 2 num_sanity_val_steps: 0 val_check_interval: 200 enable_progress_bar: true precision: 16-mixed

checkpoint: save_last: true # save at each validation time save_top_k: -1 every_n_train_steps: 200`

I find some regular noises (like the neat square notch in red cycle) and floaters. They usually appear after 400 steps.

[bennyguo]

I think the "noise" is probably due to the empty space pruning strategy we used. The areas you marked have been accidentally pruned, so that the geometry is "carved in". You could try setting system.renderer.grid_prune=false.

The presence of the floaters is because zero123 uses a fixed white background, so that white floaters cannot be observed in training. You may try increasing system.loss.lambda_sparsity.