low psnr & appearance modeling

[sshiningg]

I ran the MLP, AGG, RobustFilter, and 3DGS experiments with the on-the-go Mountain dataset, but the PSNR values ​​I got were all around 13. Could you tell me where the problem is?

[houchenfeng]

I had similar problems with on-the-go spot datasets with disastrous results on new perspectives. psnr is only 22，compared to 25 in this paper

[lilygoli]

Could you please add details and rendering examples? I suspect it might be an issue due to dataset format.

[lilygoli]

please follow the thread in issue https://github.com/lilygoli/SpotLessSplats/issues/11 for updates. @sshiningg

[sshiningg]

please follow the thread in issue #11 for updates. @sshiningg

Thank you for your reply. Because I encountered the following problem during the reproduction process: module 'mediapy' has no attribute 'resize'，I used the resize_image() function of the mediapy library instead, however, due to my negligence, which led to the problem of the image resolution width and height reversal. But after I fixed the problem, the reproduction result on the on-the-go Mountain dataset was 19.900, which is inconsistent with the result of 22.53 in the paper.

Also, could you tell me which camera model of colmap you are running for sparse reconstruction, e.g. SIMPLE_PINHOLE, SIMPLE_RADIAL, etc., and does the choice of camera model have any effect on the results?

Here are the GT and rendered image，I turned on the app_opt and ubp options

But after I turn on the app_opt option, I get the following error:

File “/SpotLessSplats/examples/spotless_trainer.py”, line 961, in reset_sh self.splats[“shN”], max=torch.abs(torch.tensor(value)).item() File “/anaconda3/envs/spotlessSplats/lib/python3.10/site-packages/torch/nn/modules/container.py”, line 738, in getitem return getattr (self, attr) File “/anaconda3/envs/spotlessSplats/lib/python3.10/site-packages/torch/nn/modules/module.py”, line 1614, in getattr raise AttributeError(“‘{}’ object has no attribute ‘{}’”.format( AttributeError: 'ParameterDict' object has no attribute 'shN'

In the reset_sh() function, since shN is not initialized, I skipped the step colors = torch.clamp(self.splats["shN"], max=torch.abs(torch.tensor(value)).item()), and the experiment works fine. Could this cause any issues with the results?

[lilygoli]

I think it might be coming from inconsistent cameras, or too high a threshold on the ubp. Would you please download the dataset from the link shared in #11 and see if that solves the issue? In my COLMAP setup, the camera model is set to OPENCV. We did not test with the appearance optimization of gsplat ever enabled, and the results reported in the README file are without appearance optimization (the internal gsplat appearance optimization is inconsistent with the appearance model in the paper), so I don't recommend setting it to true. However the reset_sh() function has little effect on the results so if you need to comment it out, it should be fine.

[sshiningg]

I think it might be coming from inconsistent cameras, or too high a threshold on the ubp. Would you please download the dataset from the link shared in #11 and see if that solves the issue? In my COLMAP setup, the camera model is set to OPENCV. We did not test with the appearance optimization of gsplat ever enabled, and the results reported in the README file are without appearance optimization (the internal gsplat appearance optimization is inconsistent with the appearance model in the paper), so I don't recommend setting it to true. However the reset_sh() function has little effect on the results so if you need to comment it out, it should be fine.

Thank you very much for your answer! I will follow up with experiments using the newly uploaded dataset. I'm a bit confused between the internal gsplat appearance optimization and the appearance model described in the paper. Which part of the code reflects the appearance model in the paper?

[lilygoli]

The appearance model described in the paper is not re-implemented here. We will add it in later updates.

[sshiningg]

The appearance model described in the paper is not re-implemented here. We will add it in later updates.

Thank you very much! Looking forward to your updates👀