search

tangtaogo / lidar-nerf

LiDAR-NeRF: Novel LiDAR View Synthesis via Neural Radiance Fields
https://tangtaogo.github.io/lidar-nerf-website/
MIT License
118 stars 8 forks source link

How to use a trained model to generate temporally-consistent outputs? #6

Open Tumiz opened 3 months ago

Tumiz commented 3 months ago

I have already trained a model following the instructions in the project readme. I want to use it to predict point clouds according to given view points, but I do not know how to use it. Can you give me some instructions?

tangtaogo commented 3 months ago

Hello, The key to novel views is to generate novel poses, you can update lidar poses using the given view points at https://github.com/tangtaogo/lidar-nerf/blob/8083a1d74eef6e7dc91c65ed7aedc2b45a39f76c/preprocess/kitti360_to_nerf.py#L137 and then inference the trained model on generated novel poses.

Tanfuchy commented 2 months ago

I downloaded the pre-training weight file you provided, but I didn't find the parser.add_argument about loading weight in main_lidarnerf.py, and your readme file doesn't seem to provide a command for reasoning. Besides, should root_path: str = "data/kitti360 "in nerfmvl_dataset.py be" data/nerf_mvl "?

tangtaogo commented 2 months ago

I downloaded the pre-training weight file you provided, but I didn't find the parser.add_argument about loading weight in main_lidarnerf.py, and your readme file doesn't seem to provide a command for reasoning. Besides, should root_path: str = "data/kitti360 "in nerfmvl_dataset.py be" data/nerf_mvl "?

The load_checkpoint function is in https://github.com/tangtaogo/lidar-nerf/blob/8083a1d74eef6e7dc91c65ed7aedc2b45a39f76c/lidarnerf/nerf/utils.py#L662 You can modify this logic or create your own workspace directory and place the models in it.