I attempted to train a LeRF model using Blender synthetic data, such as the Lego scene. However, it exhibited overfitting to the training views and failed to converge on novel views. You can see the result in this image:
I then found a issue that is related here, which says that Nerfacto doesn't work with blender data in the default setting, and it gives a solution. I followed the arguments it used and try to train a Nerfacto and it converged. I then tried the same argument to train a LeRF and it generated pure white:
To address this issue, I made some modifications to the LeRF code, detaching the CLIP and DINO features. It also worked:
I guess the problem may be related to feature training. It's possible that there needs to be some background density for the model to learn the features, while the RGB training may discourage background density.
Have you tried this training setting before, and is it possible to resolve the issue by adjusting certain training options?
Hi,
I attempted to train a LeRF model using Blender synthetic data, such as the Lego scene. However, it exhibited overfitting to the training views and failed to converge on novel views. You can see the result in this image:
I then found a issue that is related here, which says that Nerfacto doesn't work with blender data in the default setting, and it gives a solution. I followed the arguments it used and try to train a Nerfacto and it converged. I then tried the same argument to train a LeRF and it generated pure white:
To address this issue, I made some modifications to the LeRF code, detaching the CLIP and DINO features. It also worked:
I guess the problem may be related to feature training. It's possible that there needs to be some background density for the model to learn the features, while the RGB training may discourage background density.
Have you tried this training setting before, and is it possible to resolve the issue by adjusting certain training options?
Thank you very much!