search

Kai-46 / nerfplusplus

improves over nerf in 360 capture of unbounded scenes
BSD 2-Clause "Simplified" License
911 stars 101 forks source link

Training results in NaN losses on custom data set #58

Closed kaikostic closed 9 months ago

kaikostic commented 9 months ago

Hi! I am trying to train on my own dataset and it have been giving me a NaN result. Ive tried debugging and landed on that the fg_raw and everything after is where the problem is. I followed all the other issues that was linked with the normalization of the camera poses but it still results in NaN.

Stepping into the function that fg_raw is taken from the base=self.base_layer0 (https://github.com/Kai-46/nerfplusplus/blob/ebf2f3e75fd6c5dfc8c9d0b533800daaf17bd95f/nerf_network.py#L127) throws the first NaN.

Is this still related to the normalisation or is it could it be connected to something else? Thank you for the model, got it up and running on the test data!

kaikostic commented 9 months ago

I was just idiotic and did not doublecheck the most fundamental part...