This project is Pose Estimation on iOS with TensorFlow Lite.
If you are interested in iOS + Machine Learning, visit here you can see various DEMOs.

2D pose estimation in real-time	3D pose estimation

Features

• [x] Support 2D pose estimaiton TFLite models
  • [x] Real-time demo with Metal framwork
  • [x] Photo album demo
• [x] Support 3D pose estimation TFLite model
  • [x] Real-time demo with Metal framwork (but realtime model is not ready yet)
  • [x] Real-time pose matching demo
  • [x] Photo album demo
• [x] Render the result keypoints of 2D pose estimation in 2D demo page
• [x] Render the result keypoints of 3D pose estimation with SceneKit
• [x] Render the heatmaps of 2D pose estimation output
  • [x] Part Confidence Maps for typical heatmap based models
  • [x] Part Affinity Fields for OpenPose (2D multi-person)
• [x] Implemented pose-matching with cosine similiarity in 3D pose demo
• [x] Implemented to fix the shoulders' keypoints in 3D pose estimation to pre-process for pose-matching

Models

Joint Samples

Model Names	gif	img-0	img-1	img-2
PoseNet	-
PEFM CPM	-
PEFM Hourglass	-
OpenPose (multi-person)	-

Heatmap-ConfidenceMap Samples

Model Names	gif	img-0	img-1	img-2
PoseNet	-	-	-	-	-
PEFM CPM	-
PEFM Hourglass	-
OpenPose (multi-person)	-

Heatmap-PAF Samples

Model Names	gif	img-0	img-1	img-2
OpenPose (PAF x)	-
OpenPose (PAF y)	-

Meta Data

2D

3D

Requirements

• Xcode 11.3+
• iOS 11.0+
• Swift 5
• CocoaPods

  gem install cocoapods

Build & Run

1. Install dependencies with cocoapods

   cd ~/{PROJECT_PATH}
   pod install

2. Open the PoseEstimation-TFLiteSwift.xcworkspace and run the project

See also

• motlabs/awesome-ml-demos-with-ios
• TensorFlow Lite or Tensorflow models provided by:
  • CPM and Hourglass model provided by edvardHua/PoseEstimationForMobile
  • PoseNet model provided by tensorflow/examples
  • OpenPose model provided by ildoonet/tf-pose-estimation
  • Various model provided by osmr/imgclsmob
  • 3DMPPE PoseNet model provided by mks0601/3DMPPE_POSENET_RELEASE
• Pose estimation with Core ML - tucan9389/PoseEstimation-CoreML

License

This repository is licensed under Apache 2.0. Full license text is available in LICENSE.