Question regarding coord system and ego correction

[richardkxu]

Hi, thank you for opensource the repo! I have a question regarding the coordinate system of the input pc1-pc2 pair to train neural prior. According to line 159-161 of the preprocess_sf_nuscenes.py,

It seems like the the relative_pose_1_2 convert inbox_pc1 (which is in ego1 coord frame) to inbox_pc1_t (ego2 frame). Does this mean the GT flow (training objective for neural prior) is the scene flow from ego1 to ego2, which include both the ego motion flow and surround object movement flow? Is there a design choice reason not using the global coord or applying ego correction before generating GT flow for training neural prior?

Thank you so much!

[Lilac-Lee]

Hi, thanks for reaching out. First, there is no training involved in the neural prior. Our method is a runtime optimization-based scene flow estimation. The flow (GT flow) is a collection of all movements in the 3D scene. In the autonomous driving scenes, this flow (movement) includes both the ego motion of the AV (rigid part) and movements of the foreground (non-rigid part, e.g., cars, pedestrians, cyclists, etc.). We use neural prior to optimize non-rigid flow including background and foreground motions. However, in AV scenarios where ego motions can be obtained, removing ego motions from the scene might help with the accuracy and the speed of the optimization. Cheers.

[richardkxu]

Hi, thank you for your reply! To clarify, for points inside 3d bboxes, the GT flow includes both the ego/rigid flow and nonrigid flow; for points outside the bboxes, the GT flow includes just the ego motion flow. Is this correct?

I also have a question regarding the ICP used here: https://github.com/Lilac-Lee/Neural_Scene_Flow_Prior/blob/c820d9e06f6a4cb7e70f71edf8835a67ac6a822d/preprocess_sf_nuscenes.py#L199 Do you use ICP just in case the ego poses are not precise?

[Lilac-Lee]

Hi, Yes, the GT flow includes both the ego motion from the autonomous vehicle and the non-rigid flow. The ICP in the GT generation is somehow unnecessary since we already have accurate ego poses. But yes, it is here to correct the pose in case it is not that accurate. Cheers.

[richardkxu]

thanks for the clarification!