search

autonomousvision / sdfstudio

A Unified Framework for Surface Reconstruction
Apache License 2.0
1.97k stars 183 forks source link

How to reproduce for little views using monosdf ？ #171

Open guangxuwang opened 1 year ago

guangxuwang commented 1 year ago

Describe the bug How to reproduce for four views using monosdf ？My custom dataset is about fish in underwater scene.

Custom datasets is shown as follow： image

To Reproduce Command: ns-train monosdf --pipeline.model.sdf-field.use-grid-feature True --pipeline.model.sdf-field.hidden-dim 256 --pipeline.model.sdf-field.num-layers 2 --pipeline.model.sdf-field.num-layers-color 2 --pipeline.model.sdf-field.use-appearance-embedding True --pipeline.model.sdf-field.geometric-init True --pipeline.model.sdf-field.inside-outside True --pipeline.model.sdf-field.bias 0.8 --pipeline.model.sdf-field.beta-init 0.1 --pipeline.datamanager.train-num-images-to-sample-from 1 --pipeline.datamanager.train-num-times-to-repeat-images 0 --trainer.steps-per-eval-image 5000 --pipeline.model.background-model none --experiment-name monosdf-htnt-scan1 --pipeline.model.mono-depth-loss-mult 0.001 --pipeline.model.mono-normal-loss-mult 0.01 --pipeline.datamanager.train-num-rays-per-batch 2048 --machine.num-gpus 1 sdfstudio-data --data /model/2023/wangguangxu/sdfstudio-master/data/0711/sdf2 --include_mono_prior True --skip_every_for_val_split 30

However，the mesh results is so worse. image

niujinshuchong commented 1 year ago

Hi, could yo try to disable feature grids by --pipeline.model.sdf-field.use-grid-feature False?

guangxuwang commented 1 year ago

Hi, could yo try to disable feature grids by --pipeline.model.sdf-field.use-grid-feature False? @niujinshuchong Thank you for your reply! There are some improvements now. Can you give me more advice on the reconstruction of the sparse view?

image

niujinshuchong commented 1 year ago

Hi, now your results looks very similar to one of the failure cases we shown in MonoSDF paper. Maybe tuning the near and far plane could results in better results but it is not a general solution. Do you maybe have sparse point could from camera calibrations? It can be used to estimate a rough scale of the mono depth map and then use the mono depth as a metric depth after scale recovery.

elenacliu commented 1 year ago

so how to define the near and far plane in my meta_data.json?