search

PJLab-ADG / neuralsim

neuralsim: 3D surface reconstruction and simulation based on 3D neural rendering.
MIT License
590 stars 31 forks source link

Posed monocular rgb as input #8

Closed awarebayes closed 1 year ago

awarebayes commented 1 year ago

Hello, thanks for your awesome work. I want to apply it to posed monocular indoor scenes (some pretty large) (retail stores, my own dataset). I have images and camera poses extracted with colmap (cvg/hloc) What is the easiest way to do it? Which architecture should I try first?

ventusff commented 1 year ago

Hi, I have just updated something you might need.

You could use dataio.colmap_dataset.COLMAPDataset in this case, with docs written here.

As a starting point, I suggest you follow the monocular priors extraction process in the above link.

Then you can try with this experimental config yaml.

You should carefully estimate the center and scale (bounding_size) of each of your dataset. I have provided some automatic tools in nr3d_lib.geometry.normalize_views, but they are mostly meant for object-centric datasets and not for indoor datasets.