asus4 / tf-lite-unity-sample

TensorFlow Lite Samples on Unity
834 stars 249 forks source link

Expose mediapipe config option "min_pose_detection_confidence" #309

Closed Ale1 closed 10 months ago

Ale1 commented 1 year ago

Is your feature request related to a problem? Please describe. The pose landmark detection model seems to be using the default value of the param "min_pose_detection_confidence". The effects of this value can be seen here: https://mediapipe-studio.webapps.google.com/demo/pose_landmarker. And the documentation for this param here: https://developers.google.com/mediapipe/solutions/vision/pose_landmarker#configurations_options.

We find this value (of 0.5f) to be too permissive, as its easy to "trick" the model into detecting random hands waving in front of the camera as a person. Using the Web SDK of blaze pose with a min_pose_detection_confidence of > 0.7f seems to work better with way fewer false positives.

Describe the solution you'd like It would be ideal for this blaze pose param to be exposed in the Blazepose.Options.PoseLandmarkDetect.Options as a public float.

Describe alternatives you've considered We tried playing with the Score threshold that is part of the Options, but it only affects the drawing rather than the detection itself. Also, we tried to change the "Score" filter on the landmarks (currently hardcoded to filter out <0.2. However, even bad quality inputs are being delivered with scores above 0.7, so its hard to to use this method to segregate bad and good quality inputs.

Additional context

Here is example of "min_pose_detection_confidence" scores set at default 0.5 and then at 0.8. With lower score, its easy to trick the detection model into thinking an arm is a person laying horizontally, with fingers being detected as limbs and wrist as shoulders. With score of 0.8, we can filter those cases out.

Screenshot 2023-06-08 at 10 22 03 Screenshot 2023-06-08 at 10 22 14

SusheelNath commented 1 year ago

Hi @asus4, to also add to the above, @Ale1 and I tried that on multiple devices (iPad 2021 12.9, iPhone 12 Pro, Samsung Galaxy S10 to name a few) and this occurs on all.

stale[bot] commented 10 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.