maxbbraun / facecam

Faux 3D perspective with face tracking
MIT License
3 stars 2 forks source link

FaceCam

This Android app renders a 3D scene (made with Unity) using face tracking to move a virtual camera.

screen recording

The Android Studio and Unity projects can be imported from the android and unity directories.

Basic flow:

  1. In FaceCamera.java, detect the 2D position and size of a face using the front-facing camera.
  2. In FaceCamera.java, use OpenCV to reconstruct the face position in 3D space.
  3. In UnityPlayerActivity.java, serialize the camera pose using camera_pose.proto and send it to the Unity scene.
  4. In MoveCamera.cs, deserialize the pose and apply it to the scene's camera.

The 3D model in the scene is Coral Reef by Ashley Alicea.