bad result on custom colmap data

[huang997733]

Hi,

I found that the result of neus-colmap is really bad. I used colmap on nerf_synthetic data then trained the model, and the result is very bad.

• colmap on nerf_synthetic/chair, radius: 1

I have also tried to change radius in neus-colmap.yaml file as in #20. I have tried 0.5, 1, 1.5 which all gave me bad results.

• colmap on nerf_synthetic/chair, radius: 0.5

  

• colmap on nerf_synthetic/chair, radius: 1.5

  

I tested nerf-colmap, which seems to be working, but neus-colmap isn't working on the nerf_synthetic data. Any idea what went wrong here?

Thanks

[bennyguo]

Could you please make sure that dataset.use_mask is set to false and system.loss.lambda_mask is set to 0? If dataset.use_mask=true and you don't provide a masks folder, the object masks loaded would be all white... I'll change this behavior soon.

[huang997733]

I set dataset.use_mask to false and system.loss.lambda_mask to 0, still giving me bad results. Any other ideas on what could go wrong here?

[bennyguo]

I just changed the logic about using masks. If you are training on NeRF-Synthetic data with COLMAP poses, you should use configs/neus-colmap.yaml and do the following:

• extract masks from images and store them in a masks folder alongside with images
• set system.loss.lambda_mask=0.1
• set model.radius=0.5

[huang997733]

Thanks a lot!

[TianyuHuang-000]

I just changed the logic about using masks. If you are training on NeRF-Synthetic data with COLMAP poses, you should use configs/neus-colmap.yaml and do the following:

• extract masks from images and store them in a masks folder alongside with images
• set system.loss.lambda_mask=0.1
• set model.radius=0.5

Thanks for the update Does it mean that the colmap data has to be used with a mask folder as compulsory element? Before I can run neus_colmap on the "room" scene from mip360 dataset, but now it doesn't work. Some scenario may target to reconstruct the background surface as well and hard to give a mask, any work could be done on that?

[bennyguo]

No, you don't need to train COLMAP data with masks. Actually, the main point for implementing a background model is to enable training without masks on real-captured scenes.

Which version did you use to train on the room scene before? Did you enlarge model.radius? It is a special case as no foreground object is presented in the scene. I can get reasonable results by setting model.radius=5 (a large value) in the latest code.

[A-pril]

I meet this problem too. I only can use nerf-colmap, the test result is very bad and I can't get a obj file(the obj file is 0 KB).The test scene is a room , and I take photos by myself. I don't have masks. I have tried radius with 1 and 0.5 and neither is good. test/psnr is 5-7.

• The very bad result

• The relative good result

Do u have any advice on this issue? maybe the param values is not correct?

[bennyguo]

@A-pril Could you try the latest code (with a bugfix on camera normalization)? See https://github.com/bennyguo/instant-nsr-pl/issues/55 for more discussions.