search

bmild / nerf

Code release for NeRF (Neural Radiance Fields)
http://tancik.com/nerf
MIT License
10k stars 1.39k forks source link

General understanding of why this works better than general optimization #79

Open mbcel opened 4 years ago

mbcel commented 4 years ago

Really interesting work!

There is one question that came up to me about this general approach of modelling a scene with an MLP compared to a general normal optimization without using deep learning.

Since the MLP that is used here only takes xyz position and direction and then is "overfitted" on the trained szene, why is this more powerful than a general (complex) look-up table? I mean the network is not really learning any general model but instead just tries to find appropiate density and color values for every location/direction, right?

So this seems like something that could be solved equally well with some general optimization and a lookup table or high dimensional vector that optimizes for best fitting colors/densities.

What is so much better by using an MLP here, because the MLP seems only to be a "fancy" lookup table in this case?

zqbai-jeremy commented 3 years ago

Really interesting work!

There is one question that came up to me about this general approach of modelling a scene with an MLP compared to a general normal optimization without using deep learning.

Since the MLP that is used here only takes xyz position and direction and then is "overfitted" on the trained szene, why is this more powerful than a general (complex) look-up table? I mean the network is not really learning any general model but instead just tries to find appropiate density and color values for every location/direction, right?

So this seems like something that could be solved equally well with some general optimization and a lookup table or high dimensional vector that optimizes for best fitting colors/densities.

What is so much better by using an MLP here, because the MLP seems only to be a "fancy" lookup table in this case?

Maybe storage space can be one benefit? There seems to be only a small number of MLP weights stored for one scene.

HolmesShuan commented 3 years ago

Since the MLP that is used here only takes xyz position and direction and then is "overfitted" on the trained scene, why is this more powerful than a general (complex) look-up table?

In this case, Overfitted seems to have considerable merit since the input includes only xyz and direction without density or RGBA information. That is the learnable parameter has to be scene-dependent.