Wu Y, Zhang Y, Zhu D, et al. EAO-SLAM: Monocular Semi-Dense Object SLAM Based on Ensemble Data Association[C]//2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020: 4966-4973. [Paper] [Arxiv] [YouTube] [bilibili] [Project page].
Extended Work
Wu Y, Zhang Y, Zhu D, et al. Object SLAM-Based Active Mapping and Robotic Grasping[C]//2021 International Conference on 3D Vision (3DV). IEEE, 2021: 1372-1381. [Paper] [Arxiv] [Project page].
If you use the code in your academic work, please cite the above paper.
1. Prerequisites
Prerequisites are the same as semidense-lines. If compiling problems met, please refer to semidense-lines and ORB_SLAM2.
The code is tested in Ubuntu 16.04, opencv 3.2.0/3.3.1, Eigen 3.2.1.
2. Building
chmod +x build.sh
./build.sh
3. Examples
3.0 We provide a demo that uses the TUM rgbd_dataset_freiburg3_long_office_household sequence; please download the dataset beforehand. The offline object bounding boxes are in data/yolo_txts folder.
3.1Object size and orientation estimation.
use iForest and line alignment:
./Examples/Monocular/mono_tum LineAndiForest [path of tum fr3_long_office]
only use iForest:
./Examples/Monocular/mono_tum iForest [path of tum fr3_long_office]
without iForest and line alignment:
./Examples/Monocular/mono_tum None [path of tum fr3_long_office]
3.2Data association
without data association:
./Examples/Monocular/mono_tum NA [path of tum fr3_long_office]
data association by IoU only:
./Examples/Monocular/mono_tum IoU [path of tum fr3_long_office]
data association by Non-Parametric-test only:
./Examples/Monocular/mono_tum NP [path of tum fr3_long_office]
data association by our ensemble method:
./Examples/Monocular/mono_tum EAO [path of tum fr3_long_office]
3.3The full demo on TUM fr3_long_office sequence:
./Examples/Monocular/mono_tum Full [path of tum fr3_long_office]
If you want to see the semi-dense map, you may have to wait a while after the sequence ends.
Since YOLO (which was not trained in this scenario) made a lot of false detections at the start of the sequence, so we adopted a stricter elimination mechanism, which resulted in the deletion of many objects at the start.
4. Videos
More experimental results can be found on our project page.
This is an incomplete version of our paper. If you want to use it in your work or with other datasets, you should prepare the offline semantic detection/segmentation results or switch to online mode. Besides, you may need to adjust the data association strategy and abnormal object elimination mechanism (We found the misdetection from YOLO has a great impact on the results).
Mur-Artal R, Tardós J D. Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras[J]. IEEE Transactions on Robotics, 2017, 33(5): 1255-1262. PDF, Code
Yang S, Scherer S. Cubeslam: Monocular 3-d object slam[J]. IEEE Transactions on Robotics, 2019, 35(4): 925-938. PDF, Code
He S, Qin X, Zhang Z, et al. Incremental 3d line segment extraction from semi-dense slam[C]//2018 24th International Conference on Pattern Recognition (ICPR). IEEE, 2018: 1658-1663. PDF, Code
@inproceedings{wu2020eao,
title={EAO-SLAM: Monocular semi-dense object SLAM based on ensemble data association},
author={Wu, Yanmin and Zhang, Yunzhou and Zhu, Delong and Feng, Yonghui and Coleman, Sonya and Kerr, Dermot},
booktitle={2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
pages={4966--4973},
year={2020},
organization={IEEE}
}
@inproceedings{wu2021object,
title={Object SLAM-Based Active Mapping and Robotic Grasping},
author={Wu, Yanmin and Zhang, Yunzhou and Zhu, Delong and Chen, Xin and Coleman, Sonya and Sun, Wenkai and Hu, Xinggang and Deng, Zhiqiang},
booktitle={2021 International Conference on 3D Vision (3DV)},
pages={1372-1381},
year={2021},
organization={IEEE}
}