search

sarafridov / plenoxels

JAX implementation of Plenoxels
111 stars 24 forks source link

interpolating small negative sigmas #9

Closed jenkspt closed 2 years ago

jenkspt commented 2 years ago

Hi I'm wondering about the trilinear interpolation of sigmas. From my understanding going from sigma to alpha looks like this: α = 1 - exp(-max(sum(σᵢ * wᵢ), 0))) where σᵢ, wᵢ are the neighbor sigmas and weights for trilinear interpolation.

I think it's possible for the sigmas to take on small negative values -- which could possibly cancel out other small positive sigmas. For example consider the 8 neighboring sigmas with values ([-0.001, -0.001, -0.001, -0.001, -0.001, -0.001, -0.001, 0.007]) and weights = 1/8 -- the result is a interpolated sigma of zero even though there are non-zero sigmas in the neighborhood.

Applying the relu function to sigmas before interpolating would fix this problem -- if it is actually a problem. Maybe the the effect is negligible?

sarafridov commented 2 years ago

In the rendering formula each sample gets assigned an alpha = 1.0 - jnp.exp(-jax.nn.relu(sigma) * dists), so the relu is around the density sigma associated with each sample along the ray. As you point out this sigma is derived by interpolating the neighboring values, which are not constrained to be positive, so positive and slightly negative neighbors could potentially cancel out. This isn't a problem, and it may even be a benefit because it allows the grid a little more flexibility in where exactly the interpolated density crosses zero (and will never actually try to render a negative density). Your suggestion of moving the relu to before the interpolation also works fine, but will result in slightly different results.

jenkspt commented 2 years ago

Thanks for your insight!