CLOCs: Camera-LiDAR Object Candidates Fusion for 3D Object Detection. CLOCs is a novel Camera-LiDAR Object Candidates fusion network. This is my implementation based on Open-PCdet and MMDetection. The paper information can be found below.
@article{pang2020clocs,
title={CLOCs: Camera-LiDAR Object Candidates Fusion for 3D Object Detection},
author={Pang, Su and Morris, Daniel and Radha, Hayder},
booktitle={2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
year={2020}
organization={IEEE}
}
Thanks to the original implementation, it helps me a lot.
numpy
torch
kornia==0.5.2
numba
pyyaml
easydict
Add the CLOCs directory to your PYTHONPATH, just add below line to your ~/.bashrc
file:
export PYTHONPATH=$PYTHONPATH:'/dir/to/your/CLOCs/'
All data you need can be downloaded from here(or Googledrive).
new 40 recall points
The code is partly based on Open-PCdet, you need to install it first to get 3D detection data. Or you can just use data from here(Googledrive) without PCdet. Please follow the step here. Please remind that the PCdet should be installed in another folder not in this folder.
You can just run
python setup.py develop
OpenPCDet/tools/eval_utils/eval_utils.py
line 61: annos = dataset.generate_prediction_dicts(
batch_dict, pred_dicts, class_names,
output_path=final_output_dir if save_to_file else None
)
det_annos += annos
Here, annos
denotes one detection results for one input. You can add one line torch.save(***)
or other method to save the detection results. Here you need to predict all dataset results that training and validation dataset. So that, you can train your CLOCs model and validate it.
Car -1 -1 -10 1133.50 278.19 1225.04 329.51 -1 -1 -1 -1000 -1000 -1000 -10 0.0150
Car -1 -1 -10 1156.30 225.86 1225.01 262.08 -1 -1 -1 -1000 -1000 -1000 -10 0.0139
Car -1 -1 -10 1044.50 215.57 1112.86 259.75 -1 -1 -1 -1000 -1000 -1000 -10 0.0021
Car -1 -1 -10 1166.70 225.15 1225.02 246.63 -1 -1 -1 -1000 -1000 -1000 -10 0.0014
Car -1 -1 -10 751.01 150.31 782.09 177.66 -1 -1 -1 -1000 -1000 -1000 -10 0.0014
First of all, you need to organize your 2D and 3D results as below:
.
└── clocs_data
├── 2D
│ ├── 000000.txt
│ ├── 000001.txt
│ └── 000002.txt
├── 3D
│ ├── 000000.pt
│ ├── 000001.pt
│ └── 000002.pt
Modify some parameters of the file generate_data.py
following below instruction. Or you can just input this args when you run python generate_data.py
.
'--rootpath' data dir where you save your all data
'--d2path' Name of the parent folder where the prediction results are stored, for example 'your_clocs_data_path/2D'
'--d3path' is same as above
'--infopath' is the path of the file 'kitti_infos_trainval.pkl' produced by pcdet
'--inputpath' is where the input data is stored
'--log-path' is the path where you want to store your model
if you modified above well, then just
python generate_data.py --args
then you can get the input data that stored in 'inputpath'
. Your data dir should be like below:
.
└── clocs_data(rootpath)
├── 2D(d2path)
│ ├── 000000.txt
│ ├── 000001.txt
│ └── 000002.txt
├── 3D(d3path)
│ ├── 000000.pt
│ ├── 000001.pt
│ └── 000002.pt
└── input_data
├── 000000.pt
├── 000001.pt
└── 000002.pt
You should organize your data dir as below:
.
└── clocs_data
├── 2D
│ ├── 000000.txt
│ ├── 000001.txt
│ └── 000002.txt
├── 3D
│ ├── 000000.pt
│ ├── 000001.pt
│ └── 000002.pt
├── index
│ ├── train.txt
│ ├── trainval.txt
│ └── val.txt
├── info
│ ├── kitti_infos_trainval.pkl
│ └── kitti_infos_val.pkl
└── input_data
├── 000000.pt
├── 000001.pt
└── 000002.pt
Here, the index dir is copied from ./index
and the files in info
are got from pcdet or you can download from here(Googledrive)
train.py
same as the generate_data.py
. Thenpython train.py
eval.py
as above rule. thenpython eval.py
It will validate all your model, you can choose the best one.
baseline.py
as above rule.python baseline.py
This implementation just store the input data, 3D detection data and 2D detection data in disks, so the training process is much faster than original CLOCs implementation. If you want to use CLOCs in other dataset or other detection method, you just need to modify the data pre-processing or just output the detection results as here defined. By following above, you can apply this fusion network in any method you want and it can be trained well in a short time.
Our codes are inspired by CLOCs a lot and are based on PCDet and MMdetection. Thanks for their excellent work.