search

google-ai-edge / mediapipe-samples

Apache License 2.0
1.49k stars 382 forks source link

how to combine a custom model to default one? #74

Closed fan123199 closed 1 year ago

fan123199 commented 1 year ago

I have follow the page https://developers.google.com/mediapipe/solutions/vision/gesture_recognizer/customize and get a custom model gesture_recognizer.task that can recognize rock paper scissors.

And I found Android SDK can only load one model. So how can I use this model conbime with the default one https://developers.google.com/mediapipe/solutions/vision/gesture_recognizer#models

jenperson commented 1 year ago

I'm not completely sure what you're asking, but I'll give a couple answers and you can let me know if one of them answers your question.

If you're trying to create a solution that detects the gestures from the gesture classification model bundle as well as rock, paper, and scissors, then you need to include all of those gestures when training your gesture recognizer with MediaPipe Model Maker. You should provide images and classifications for any gestures you want to include--the default gestures from the provided model bundle aren't included when training a custom gesture recognizer.

If you want to add your rock paper scissors model bundle to an Android app, then follow the directions for creating the task and set the model asset path to the location of your custom gesture recognizer.

fan123199 commented 1 year ago

Thank you for you answer. Actualy the first answer is what I want.

Because my real purpose is that my demo app can regconize rock paper scissors , and also "👍, 👎, ✌️, ☝️, ✊, 👋, 🤟" at same time.

As you answer, I need to train all gesture from images. So How could I get the source dataset of default demo model here https://developers.google.com/mediapipe/solutions/vision/gesture_recognizer#models ?

I thought it would be offered by mediapipe official. So that can let us easly use the hand guesture model.

PaulTR commented 1 year ago

So I haven't been able to get an exact answer on where our data for the model came from, but I'd really recommend checking out Kaggle for datasets that would be relevant. This one in particular looks like it might have the gestures you're wanting that can be mixed in with the rock paper scissors data.