search

bennyguo / instant-nsr-pl

Neural Surface reconstruction based on Instant-NGP. Efficient and customizable boilerplate for your research projects. Train NeuS in 10min!
MIT License
856 stars 84 forks source link

Problem with psnr and ssim #53

Open lidemina opened 1 year ago

lidemina commented 1 year ago

Hi @bennyguo，thank you for your excellent work. During the experiment, I found that the psnr and ssim is very low, even lower than nues, and then I found that the perspective of the rendered image and the original image are different, and the two images are not aligned. ngp0 ssim

bennyguo commented 1 year ago

Hi! Did you use the camera parameters provided by BlendedMVS or you estimated them by COLMAP?

lidemina commented 1 year ago

thanks for your reply! I used camera parameters estimated by colmap

bennyguo commented 1 year ago

This is probably because the estimated poses are different from those provided by BlendedMVS, causing the misalignment. You could try implementing a new DataModule utilizing the BlendedMVS data, or wait for my implementation :) If you are interested in implementing it on your own, you may refer to datasets/colmap.py for an example. All you have to do is to provide self.directions, self.all_c2w, self.all_images, and self.all_fg_masks in the dataset.

bennyguo commented 1 year ago

@lidemina Hi! I just found a bug in the COLMAP dataset. The camera parameters are inconsistent in the training and validation datasets due to the randomness of RANSAC used to estimate the ground plane, which leads to low PSNR as you described. I'll push a fix later today.

lidemina commented 1 year ago

@lidemina Hi! I just found a bug in the COLMAP dataset. The camera parameters are inconsistent in the training and validation datasets due to the randomness of RANSAC used to estimate the ground plane, which leads to low PSNR as you described. I'll push a fix later today.

great to hear that, thanks for your reply!