Arvis

This is a project to control 3D objects by hand in augmented reality.

Generate virtual hand

Computer vision technology recognizes human hands and creates virtual hands within Unity.

Extract coordinates of hands

To make the virtual hand as above, we need to extract the coordinates of the hand. So we need to extract the hand coordinates. And To extract it, the following process is required.

1. Convert to HSV image

2. Binarize the HSV image for detecting skin

3. Remove face

4. Detect contours and defects

5. Reduce the number of defects

6. Detect the center of hand

7. Find the defects of fingertips

Control AR object

After the above process, you can touch the AR object with your hands. This technology enables physical interaction with objects in the Unity virtual world. This is expected to provide a new user experience that encounters the AR environment.