How to ensure continuous deformation?

[Lancial]

Hi,

First of all, thank you for this fantastic work! This might be a question about dynamic nerf...but your paper mentioned that "per-frame deformations are captured continuously". Does this mean that the predicted scene displacement is temporally continuous or spatially continuous? Either way, I was wondering how are they achieved.

If I understand correctly, the add_tv_loss option indicates a total variation loss which encourages spatially smooth deformation. However, the motion network (G) only predicts delta_x given the original x and t, so it doesn't explicitly ensure temporal continuity, right? I appreciate your response!

[yuehaowang]

Hi, thanks for your question. The "continuous" property owes to the dynamic scene modeling, i.e., D-NeRF, which is an MLP mapping $(x,t)\rightarrow \Delta x$. Compared with previous scene modeling, e.g., discrete grids and ED graphs, MLP-based scene representations are naturally continuous. We can query geometry and appearance for any continuous space-time coordinates within the capture range. This is also reflected in the qualitative experiments, where each frame's deformation is well-tracked without abrupt changes in geometry and color, unlike the comparison method.

Actually, the TV loss is not used in our experiments despite we included some related code. I did some trials on spatial and temporal continuity regularisations but they did not significantly promote the performance. Nevertheless, if your deformable scenarios are much more challenging, I guess they may somewhat work.