YOLO-Object-Detection-for-Pick-and-Place-task-using-ROS-on-KUKA-iiwa
Robots use computer vision in many operations. In this repository, we are going to address one of these operations, pick and place, which is the most widely used in production lines and assembly processes. The aim of this project is to select 3 objects out of 6 objects in robot work environment, and place them in specific-colored regions, so that each object will be placed in colored region initially specified by the operator.
The challenges of this project go as follows:
- Object Detection: YOLOv4
- Box Detection: Color Segmentation in HSI and Morphology operations
- Pick and Place: Object center and orientation and Box center and orientation
- Localization w.r.t Robot Base: Eye-In-Hand Camera Calibration and Transformations
- Hardware Implementation: KUKA iiwa and ROS
Workplace Setup (Objects)
Dataset is attached above.
Labelling is done on roboflow
Dataset preprocessing and augmentations
YOLO Model Testing for Object Detection
- Test1
Problems arise due to illumination change, camera orientation, false positive results (detect the gripper as an object, detect background as an object).
Solution: Applying thresholding on detection confidence score.
- Test2
Object Center and Orientation
Box Detection
- Color Segmentation in HSI
- Morphology operations
- Box center and orientation
Localization w.r.t Robot Base
- Camera Intrinsics
- Eye-In-Hand Camera Calibration
- Transformations
Hardware Implementation
KUKA iiwa robot iiwa_stack
Demo
Full video is attached Demo
