search

georghess / neurad-studio

[CVPR2024] NeuRAD: Neural Rendering for Autonomous Driving
https://research.zenseact.com/publications/neurad/
Apache License 2.0
323 stars 22 forks source link

Pose estimation of Pandaset #47

Open Anji-Builds opened 1 month ago

Anji-Builds commented 1 month ago

How is the poses estimated for training in neurad method?

As far as I have understood, generally preprocessing is required for custom datasets in nerfstudio. How is it that we skip ns-process-data step and directly proceed to python nerfstudio/scripts/train.py neurad pandaset-data?

I have noticed it is also similar for the unisim training suggestion in the repo. What makes us directly skip the preprocessing step?

If someone could guide me it would be really helpful for me to understand the working.

JulienStanguennec-Leddartech commented 4 weeks ago

PandaSet sequences already has camera and lidar poses provided, so no pre-processing is required.

You can see how the camera poses are parsed here: https://github.com/georghess/neurad-studio/blob/main/nerfstudio/data/dataparsers/pandaset_dataparser.py#L173

Anji-Builds commented 3 weeks ago

@JulienStanguennec-Leddartech Are both camera and lidar poses necessary or just Camera would poses be sufficient? ie. can I omit all the parts that take in lidar values and would it still work?

atonderski commented 1 week ago

Hi, did you try this? We have not developed or tested a camera-only version of neurad, but hopefully it can be achieved with minimal code modifications