monosdf for outdoor dataset

[yxt7979]

Hi，thank you for the great work! I saw the previous issue and found this line in 'customdata2monosdf.py' should change as following: Do all outdoor scenes need to be changed when using monosdf? Hope your reply~

This line

scale_mat[:3] *= 2. / (np.max(max_vertices - min_vertices) + 3.) 

is used for ScanNet (indoor scenes) where the cameras are inside the rooms. In the DTU cases it should be changed to something like

scale_mat[:3] *= 3. / (np.max(max_vertices - min_vertices))

Originally posted by @niujinshuchong in https://github.com/autonomousvision/monosdf/issues/21#issuecomment-1279687517

[yxt7979]

The mesh I reconstructed using monosdf for outdoor custom dataset is not good enough， I'm wondering if it's the reason here

[niujinshuchong]

@yxt7979 In the outdoor scene, you need to normalise the scene such that it's bounded in a unit cube. Or you could only place the ROI region in the uni cube and use a background model for others.

We haven't try outdoor scenes before, could you share some of your images and the monocular priors?

[yxt7979]

Hi, this is the kind of pics we want to use~although the prior depth is quete good, the reconstruction result is blurry. I use the gt pose and gt intrinsic, and I imitate the data constructed as nice slam.

[yxt7979]

and the training looks to be converging

[yxt7979]

should this line should be changed like this~ scale_mat[:3] *= 3. / (np.max(max_vertices - min_vertices))

[niujinshuchong]

@yxt7979 I think in the indoor scene, you don't change it like that, because in that case the camera will be distributed from [-3, 3].

Maybe you can normalise it as

scale_mat[:3] *= 1. / (np.max(max_vertices - min_vertices))

Btw, how do you set near and far in the config? And maybe you need to tune the weight for the depth loss.

[lazyDamon]

@niujinshuchong Hi, I am also interested in the reconstruction of outdoor scenes. I find that the sky has depth and normal, when I reconstructed the outdoor scenes. Is it reasonable? Will it affect our depth loss and normal loss? If we use pixel segmentation to remove these areas or set it to inf, will it affect our loss fitting?

[niujinshuchong]

@lazyDamon Removing the rays of the sky region would definitely help.

[CZ-Wu]

@yxt7979 I try similar experiments. The geometric initialization does not work for me and I get blurry results like yours. You could try it by setting geometric_init = False.

[niujinshuchong]

@CZ-Wu Could you share some results of using geometric_init = True and False? Do you mean using geometric_init get worse results?