Neural Radiance Field training can be accelerated through the use of grid-based representations in NeRF's learned mapping from spatial coordinates to colors and volumetric density. However, these grid-based approaches lack an explicit understanding of scale and therefore often introduce aliasing, usually in the form of jaggies or missing scene content. Anti-aliasing has previously been addressed by mip-NeRF 360, which reasons about sub-volumes along a cone rather than points along a ray, but this approach is not natively compatible with current grid-based techniques. We show how ideas from rendering and signal processing can be used to construct a technique that combines mip-NeRF 360 and grid-based models such as Instant NGP to yield error rates that are 8% - 76% lower than either prior technique, and that trains 22x faster than mip-NeRF 360.
🔑 Key idea:
Migrate the idea of mip-NeRF 360's anti-aliasing to Instant NGP (iNGP).
💪 Strength:
In Figure 2, they argue that iNGP's learning function is step-wise linear and non-smooth. High-frequency bias is critical to fail to capture smooth low-frequency targets.
😵 Weakness:
Why don't we adjust other hyperparameters, e.g., grid-size exponential factor $b$.
In Section 4, interestingly, they proposed the normalized weight decay for hash entries is proposed defined as $\sum_l \text{mean}(V_l^2)$, which penalizes the sum of the mean of squared hash values at each level. The coarse scales are penalized by orders of magnitude more than fine scales. Notice that the number of elements is different since coarse scales have a number of vertices less than the hash table size. They proposed to use 0.1 for the weight of this loss. It seems to have a significant effect, as described in the ablation study of Table 1 and the corresponding text.
Furthermore, they argued that small view-dependent MLP (ablation M) for iNGP decreases accuracy.
Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields
Barron et al., arXiv 2023
🔑 Key idea:
💪 Strength:
😵 Weakness:
🤔 Confidence:
✏️ Memo: