Created by Martin Hahner at the Computer Vision Lab of ETH Zurich.
by Martin Hahner, Christos Sakaridis, Mario Bijelic, Felix Heide, Fisher Yu, Dengxin Dai, and Luc van Gool
📣 Oral at CVPR 2022.
Please visit our paper website for more details.
.
├── calib # contains the LiDAR sensor calibration file used in STF
│ └── ...
├── lib # contains external libraries as submodules
│ └── ...
├── splits # contains the splits we used for our experiments
│ └── ...
├── tools # contains our snowfall and wet ground simulation code
│ ├── snowfall
│ │ ├── geometry.py
│ │ ├── precompute.py
│ │ ├── sampling.py
│ │ └── simulation.py
│ └── wet_ground
│ ├── augmentation.py
│ ├── phy_equations.py
│ ├── planes.py
│ └── utils.py
├── .gitignore
├── .gitmodules
├── LICENSE
├── pointcloud_viewer.py # to visualize LiDAR point clouds and apply various augmentations
├── README.md
└── teaser.gif
Datasets supported by pointcloud_viewer.py:
Note:
The snowfall and wet ground simulation is only tested on the SeeingThroughFog (STF) dataset.
To support other datasets as well, code changes are required.
This software is made available for non-commercial use under a Creative Commons License.
A summary of the license can be found here.
If you find this work useful, please consider citing our paper.
@inproceedings{HahnerCVPR22,
author = {Hahner, Martin and Sakaridis, Christos and Bijelic, Mario and Heide, Felix and Yu, Fisher and Dai, Dengxin and Van Gool, Luc},
title = {{LiDAR Snowfall Simulation for Robust 3D Object Detection}},
booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
year = {2022},
}
You may also want to check out our earlier work
Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather.
@inproceedings{HahnerICCV21,
author = {Hahner, Martin and Sakaridis, Christos and Dai, Dengxin and Van Gool, Luc},
title = {{Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather}},
booktitle = {IEEE International Conference on Computer Vision (ICCV)},
year = {2021},
}
1) Install anaconda.
2) Execute the following commands.
# Create a new conda environment.
conda create --name snowy_lidar python=3.9 -y
# Activate the newly created conda environment.
conda activate snowy_lidar
# Install dependencies.
conda install matplotlib pandas plyfile pyaml pyopengl pyqt pyqtgraph scipy scikit-learn tqdm -c conda-forge -y
pip install PyMieScatt pyquaternion
# Clone this repository (including submodules!).
git clone git@github.com:SysCV/LiDAR_snow_sim.git --recursive
cd LiDAR_snow_sim
3) If you want to use our precomputed snowflake patterns, you can download them (2.3GB) as mentioned below.
wget https://www.trace.ethz.ch/publications/2022/lidar_snow_simulation/snowflakes.zip
unzip snowflakes.zip
rm snowflakes.zip
4) If you want to use DROR as well,
you need to install PCL or download the point indices (215MB) as mentioned below.
wget https://www.trace.ethz.ch/publications/2022/lidar_snow_simulation/DROR.zip
unzip DROR.zip
rm DROR.zip
5) Enjoy pointcloud_viewer.py.
python pointcloud_viewer.py
6) If you also want to run inference on the STF dataset, a couple of extra steps are required.
Note: For unknown reasons, this can roughly slow down the augmentation(s) by a factor of two.
# Download our checkpoints (265MB)
wget https://www.trace.ethz.ch/publications/2022/lidar_snow_simulation/experiments.zip
unzip experiments.zip
rm experiments.zip
# Install PyTorch.
conda install pytorch==1.10.1 torchvision==0.11.2 torchaudio==0.10.1 cudatoolkit=11.3 -c conda-forge -c pytorch -y
# Install spconv
pip install spconv-cu113
# build pcdet
cd lib/OpenPCDet
python setup.py develop
cd ../..
The code has been successfully tested on
Please feel free to suggest improvements to this repository.
We are always open to merge useful pull request.
This work is supported by Toyota via the TRACE project.
The work also received funding by the AI-SEE project with national funding from
We also thank the Federal Ministry for Economic Affairs and Energy for support within
VVM-Verification and Validation Methods for Automated Vehicles Level 4 and 5, a PEGASUS family project.
Felix Heide was supported by an NSF CAREER Award (2047359),
a Sony Young Faculty Award, and a Project X Innovation Award.
We thank Emmanouil Sakaridis for verifying our derivation of occlusion angles in our snowfall simulation.