search

PJLab-ADG / neuralsim

neuralsim: 3D surface reconstruction and simulation based on 3D neural rendering.
MIT License
602 stars 31 forks source link

Train Test split #6

Closed tangtaogo closed 1 year ago

tangtaogo commented 1 year ago

Hi, Thanks for your great work! Where can I find the train-test split for waymo dataset?

ventusff commented 1 year ago

Hi, For reconstruction or view synthesis tasks, usually there is no train-test split, as the camera and LiDAR data themselves are considered to be self-supervisions. The ground truth 3D shape and appearance are never revealed to the network (they don't even exist). This is also what other works in this area will assume.

Are you referring to seperating views for novel view synthesis (NVS) evaluation? I think it's not hard to make such experiments with the current codebase.

As for the sequences we select, there are detailed descriptions in StreetSurf paper's first Supp section. You can also find the selected sequences in any of the StreetSurf config yamls

tangtaogo commented 1 year ago

Yes, I mean the NVS evaluation. How can I obtain the values from the tables in the paper? May I ask how you divided a test-set for each sequence to test PSNR, chamfer distance, etc.?

ventusff commented 1 year ago

I do not split test-set for NVS in the paper. The tables in the paper are directly evaluated on the original frames used in training (which is also all the frames available). This is true for our method and all the baseline methods mentioned.

I can provide a detailed instruction about how to obtain the evaluation results exactly the same way as the paper:

  1. Assume you have already train one or more experiments. Let's say they have a parent directory /path/to/exps

NOTE: You can also use our pretrained models to do this. Remember to specify your processed waymo data root with --dataset_cfg.param.root=/path/to/your/processed/

  1. Eval appearance with --downscale=2:

    python code_single/tools/eval_directory.py --overall_dir /path/to/exps --downscale=2

    This will generate a json file in /path/to/exps/segxxx/eval, which will contain PSNR and SSIM metric results.

  2. Eval LiDAR

    python code_single/tools/eval_lidar_directory.py --overall_dir /path/to/exps --forward_inv_s=null --lidar_id=lidar_TOP --dirname=eval_lidar_top

    This will generate a json file in /path/to/exps/segxxx/eval_lidar_top, which will contain CD and RMSE metric results.

tangtaogo commented 1 year ago

Thank you for your timely and patient response!