October 14, 2022
: We've released code & models for map-view segmentation
June 9, 2022
: We've released the tech report for Geometry-guided Kernel Transformer (GKT). This work is still in progress and code/models are coming sonn. Please stay tuned! ☕️
We present a novel and efficient 2D-to-BEV transformation, Geometry-guided Kernel Transformer (GKT).
git clone https://github.com/hustvl/GKT.git
Method | Kernel | mIoU (Setting 1) | mIoU (Setting 2) | FPS | model |
---|---|---|---|---|---|
CVT | - | 39.3 | 37.2 | 34.1 | [model]() |
GKT | 7x1 | 41.4 | 38.0 | 45.6 | model |
Note: FPS are measured on one 2080 Ti GPU.
For map-view nuScenes segmentation, we mainly build the GKT based on the awesome CrossViewTransformer.
# map-view segmentation
cd segmentation
# install dependencies
pip install -r reuqirements.txt
pip install -e .
Download the pretrained model efficientnet-b4-6ed6700e.pth
mkdir pretrained_models
cd pretrained_models
# place the pretrained model here
python scripts/train.py +experiment=gkt_nuscenes_vehicle_kernel_7x1.yaml data.dataset_dir=<path/to/nuScenes> data.labels_dir=<path/to/labels>
Using the absolute path of the checkpoint is better.
python scripts/eval.py +experiment=gkt_nuscenes_vehicle_kernel_7x1.yaml data.dataset_dir=<path/to/nuScenes> data.labels_dir=<path/to/labels> experiment.ckptt <path/to/checkpoint>
python scripts/speed.py +experiment=gkt_nuscenes_vehicle_kernel_7x1.yaml data.dataset_dir=<path/to/nuScenes> data.labels_dir=<path/to/labels>
coming soon.
We sincerely appreciate the awesome repos cross_view_transformers and fiery!
GKT is released under the MIT Licence.
If you find GKT is useful in your research or applications, please consider giving us a star 🌟 and citing it by the following BibTeX entry.
@article{GeokernelTransformer,
title={Efficient and Robust 2D-to-BEV Representation Learning via Geometry-guided Kernel Transformer},
author={Chen, Shaoyu and Cheng, Tianheng and Wang, Xinggang and Meng, Wenming and Zhang, Qian and Liu, Wenyu},
journal={arXiv preprint arXiv:2206.04584},
year={2022}
}