Custom model on javascript

google-ai-edge / mediapipe

Cross-platform, customizable ML solutions for live and streaming media.

https://ai.google.dev/edge/mediapipe

Apache License 2.0

27.07k stars 5.11k forks source link

Custom model on javascript #3228

Closed elYeodeol closed 2 years ago

elYeodeol commented 2 years ago

Hey!

I'm trying to use my own custom model on API javascript mediapipe. I already saw this issues https://github.com/google/mediapipe/issues/2761 but no javascript solution has been provided.

Do you have any tips ? tuto ? documentation ? in order to do this, and make my tflite model working one mediapipe?

sureshdagooglecom commented 2 years ago

Hi @elYeodeol , Did you get a chance to check this closed issue https://github.com/google/mediapipe/issues/1741

elYeodeol commented 2 years ago

Hi,

thank you for your fast reply @sureshdagooglecom ! I read this issue and the issues related.

I'm facing this particulary error :

It seems that it doesn't really like my tflite conversion : i built a keras model and converted it with

converter.allow_custom_ops = True
converter.target_spec.supported_ops = [
  tf.lite.OpsSet.TFLITE_BUILTINS, # enable TensorFlow Lite ops.
  tf.lite.OpsSet.SELECT_TF_OPS # enable TensorFlow ops.
]

tflite_model = converter.convert()

with open('model.tflite', 'wb') as f:
    f.write(tflite_model)

Did i do the conversion right ? Maybe i am going in the wrong way and my tflite conversion is not really the issue in this...

Thank you in advance for your help

tyrmullen commented 2 years ago

Custom models are not yet supported by the JS APIs. So there's no guarantee that this process will work.

However, if you do want to try to use your own, then be aware that:

custom ops are often not supported by the graphs being used on web, so you'll probably need to remove this
SELECT_TF_OPS are not supported by our ML backends, so you'll definitely need to remove this in order to convert to .tflite format successfully
the model input parameters are hard-coded into the graph, so your model would need to match (looking at your error message, I believe the graph used here would be pose_landmark/pose_landmark_by_roi_gpu.pbtxt, so your model would need to accept 256x256 input)

elYeodeol commented 2 years ago

Thank you a lot for your reply @tyrmullen ! I have tried every ideas you have suggested me, but same error.

So there is no solution to use my custom model on the web except, maybe the android solution ?

google-ml-butler[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

google-ml-butler[bot] commented 2 years ago

Are you satisfied with the resolution of your issue? Yes No