search

sxyu / svox2

Plenoxels: Radiance Fields without Neural Networks
BSD 2-Clause "Simplified" License
2.82k stars 360 forks source link

Related Work? #24

Open ifsheldon opened 2 years ago

ifsheldon commented 2 years ago

Hi! This is an amazing work and congrats!

I find our work is somewhat related to yours and probably you may want to add our work as a related work in your paper because our idea is very similar. Our work is more in the context of scientific visualization instead of computer vision. And it's based on Taichi, which enables automatic differentiation and sparse support as well. In our work, we found that with a transfer function, even a volume with single-channel voxels is enough for a reasonable "reconstruction" (in the context of SciVis, see our demo). We didn't publish it via a paper because it started as my toy project and it wasn't our major focus.

And I've read your paper, it seems there are some differences and similarities as the following.

Technical differences:

  1. We used Taichi instead of PyTorch. Taichi differs from PyTorch in that it's kernel-based, not operator-based.
  2. We use dense volumes (although Taichi support sparse volumes), but you used sparse ones.

Theoretical differences:

  1. We use single-channel intensities (could be N-channel) in voxels and an ordinary transfer function as a representation while you use SH.
  2. We didn't add TV loss (since the results were good enough for us).
  3. We didn't do much in the context of yours. We did reconstructions on volumes whose voxel intensities can be scientifically meaningful based on our Direct Volume Rendering visualizations, while you do the reconstruction on volumes that are a representation of scenes based on photos.
  4. N-channel intensities/vector in voxels with a transfer function(a neural one or a plain learnable texture) may be consider as a superset of SH coefficients in the context of learnable representations.

Similarities:

  1. Direct Volume Rendering + Auto Differentiation
  2. Volume representation
  3. "Learning" from images (ours: rendered images, yours: photos or synthesized photos)
  4. Good convergence speed :)
sarafridov commented 1 year ago

Thanks for sharing your work! Indeed there are many related fields with similar problems and approaches. As another example, SHDOM (https://coloradolinux.com/shdom/) is a similar method to Plenoxels that is used in atmospheric modeling, also based on voxels and spherical harmonics.