Extract landmarks from tflite

[abdou31]

Step 1: Describe your environment

• Android device: Samsung A3 2016
• Android OS version: Nougat
• Google Play Services version: N/A
• Firebase/Play Services SDK version: N/A

Step 2: Describe the problem:

I have trained a Resnet model using CNN facial landmarks Github project based on Tensorflow 1.13 to detect only the eye region in real time (especially the iris region ), I have modified the project to get what I need. The result should be like this :

I have frozen the model using Tensorflow and I want to freeze the model to tflite to get a mobile model version. I have read the tutorial of ML kit firebase and I have found that support landmark detection: https://firebase.google.com/docs/ml-kit/detect-faces#example_2_face_contour_detection

• I want to know, Can I use my tflite model to detect only the region of the eye, not all the face?
• Is there any way to extract the features ( key points ) from a tflite and then drawing on the output streaming (input real-time video)? Thanks

[samtstern]

@abdou31 the link you provided is for our built-in face countours model. If you have trained your own TF model and want to use it in your app you want to follow the custom models documentation: https://firebase.google.com/docs/ml-kit/android/use-custom-models

Is that what you're looking for?

[abdou31]

I have seen this link but I want to know that if it can help me ( case of eye's detection )

[samtstern]

@abdou31 yes, MLKit will not change the capabilities of your model. If it detects eyes well, it will continue to do so.

[abdou31]

Thanks, but I really need an answer to my second question:

• Is there any way to extract the features ( key points ) from a tflite and then drawing on the output streaming (input real-time video)?

[samtstern]

@abdou31 sorry there's nothing like that built in, you have to set up the video yourself and send each frame to your custom model and then draw the output over the video yourself.

[abdou31]

Ok, I will try do that but how can I extract the prediction from the ouptput layer in tflite model file?

[samtstern]

@abdou31 sorry I really don't know about TFLite to answer that (I wish I did!) have you tried asking on StackOverflow?

[abdou31]

I have asked on StackOverflow but I didn't get any answer