Question about PoseNet Implementation on Nice slam

[JerryZhang030311]

Hi there,

I'm currently implementing your continuous-pose-in-Nerf project on co_slam and I have several questions regarding the optimization of PoseNet. Specifically, I'm wondering about the number of iterations per optimization step for each pair of PoseNet (Fi and Fo in the paper)， did you optimize the PoseNet every step during the iterations. Additionally, did you use just one set of PoseNet or multiple sets of PoseNet for joint optimization of all poses in the scene?

Thanks in advance for your assistance!

Best regards Jerry Zhang

[qimaqi]

Dear Jerry Zhang Thanks for reaching out, for every new coming frame we do exactly same thing as normal NeRF-SLAM, optimize PoseNet every step during iterations, we follow the same cconfig from the table we borrow. We only use one PoseNet for whole scene and two PoseNet for intrinsic motion experiments.

[JerryZhang030311]

Dear Jerry Zhang Thanks for reaching out, for every new coming frame we do exactly same thing as normal NeRF-SLAM, optimize PoseNet every step during iterations, we follow the same cconfig from the table we borrow. We only use one PoseNet for whole scene and two PoseNet for intrinsic motion experiments.

Thank you for your response, which has provided me with a deeper understanding of your work. I have a few more questions:

In the Nerf-SLAM experiments, is bundle adjustment still retained? I noticed in the Supplementary Material that it mentions "BA is disabled" in Implementation details in the 9th unit "Visual SLAM with Depth and IMUs”. Does this mean that bundle adjustment is not used in this work in Nerf-SLAM?

In the experiments, during the optimization process of the PoseNet neural network, how is it ensured that the neural network mapping can maintain the correctness of the representations of previous scenes when optimizing for new scenes? In other words, are there any methods employed in your work to locally optimize the neural network, ensuring that the mapping of past poses over time is not disrupted in the new optimization?

Thank you for your patient clarification.

Best regards, Jerry Zhang

[qimaqi]

Dear Jerry Zhang I disable all bundle adjustment follow the setting of the NICE-SLAM, so yes I do not run BA on all NeRF-SLAM experiments. You can understand PoseNet as another neural implicit functioin so it overfit for each scene and will not generalize to new scenes. We do it by initialize a PoseNet at beginning for every task. Hopefully this will adress all your questions.

[JerryZhang030311]

Dear Jerry Zhang I disable all bundle adjustment follow the setting of the NICE-SLAM, so yes I do not run BA on all NeRF-SLAM experiments. You can understand PoseNet as another neural implicit functioin so it overfit for each scene and will not generalize to new scenes. We do it by initialize a PoseNet at beginning for every task. Hopefully this will adress all your questions.

Dear Qi Ma， I still have some questions, if you are open to it, I would appreciate the opportunity to connect with you further, perhaps over a brief chat. If you're comfortable, we could exchange contact information such as a WeChat, or any means that you prefer. Alternatively, I understand if you prefer to keep communication within the GitHub platform.

Thank you for your time and consideration.

Best regards, Jerry Zhang

[qimaqi]

Dear Jerry Zhang

Currently Github will be better since other people can also see the discussion. And of course we can exchange contact information but I would rather not release it here publicly, we can contact through mail: qimaqi@ethz.ch for personal.

Best Qi

[heyicheng-never]

Thank you for your work and selfless contributions to open source! I have the same questions.

The first one is about the catastrophic forgetting problem in MLP. If each time only the pose of the current frame is optimized, won't the previous trajectory be forgotten? In other words, in some NeRF-SLAM approaches that only perform BA on keyframes, like co-slam, does that mean your method wouldn't work at all?

The second question is, what exactly does it mean not to run BA in NeRF-SLAM? As far as I know, NeRF-SLAM involves two threads: one for tracking, which optimizes only the pose, and another for BA, which jointly optimizes both the pose and the NeRF. When you mention not using BA, does that mean you only optimize the Nerf without joint optimization? Wouldn't this lead to significant reconstruction errors in NeRF-SLAM?