on iOS, when calling classify (a.k.a let results = try classifier.classify(audioTensor: inputAudioTensor)) on a model created using “TensorFlow Lite Model Maker”, then results.classifications[1].categories is always empty (in my case it was expected to output maximum of 5 values)
I think there’s a bug on pod “TensorFlowLiteTaskAudio” because the same model works as expected on Android.
The Android app works fine. In other words, when I execute
tensorAudio.load(recorder)
val output = classifier.classify(tensorAudio)
then:
output[0].categories has a maximum of 521 values (as expected)
ouput[1].categories has a maximum of 5 values
the image below show a screenshot of the app (left) and a debugging session(right) that shows the values mentioned above.
iOS doesn't work ❌
The iOS app doesn’t work as expected. In other words, when I execute
try inputAudioTensor.load(audioRecord: audioRecord)
let results = try classifier.classify(audioTensor: inputAudioTensor)
then:
results.classifications[0].categories has a maximum of 5 values ([the expected value is 521)
results.classifications[1].categories is always empty (the expected value is 5)
the image below show a screenshot of the app (left) and a debugging session(right) that shows the values mentioned above.
ps1: I'm using TensorFlowLiteTaskAudio 0.4.3
ps2: the iOS app works fine when using the default yamnet model. You can see in the image below that results.classifications[0].categories has a maximum of 521 values (as expected)
Question
Is there a bug on the pod “TensorFlowLiteTaskAudio”? Or is there some special configuration that should be done on iOS when integrating a model generated by “TensorFlow Lite Model Maker”? Or should I change something when generating the model .tflite for iOS?
TLDR:
on iOS, when calling classify (a.k.a
let results = try classifier.classify(audioTensor: inputAudioTensor)
) on a model created using “TensorFlow Lite Model Maker”, thenresults.classifications[1].categories
is always empty (in my case it was expected to output maximum of 5 values)I think there’s a bug on pod “TensorFlowLiteTaskAudio” because the same model works as expected on Android.
Sample repository to reproduce this bug : https://github.com/ghashi/tensor-flow-lite-task-audio-poc
Long version
I created a model using "TensorFlow Lite Model Maker Model”
I followed this tutorial https://www.tensorflow.org/lite/models/modify/model_maker/audio_classification and created a .tflite model that recognizes 5 kinds of birds songs. Here’s the versions I used to create it:
Then, I integrated this model into an Android and an iOS application. I used the tensorflow “audio classification demo” repository to do this.
Android works ✅
The Android app works fine. In other words, when I execute
then:
output[0].categories
has a maximum of 521 values (as expected)ouput[1].categories
has a maximum of 5 valuesthe image below show a screenshot of the app (left) and a debugging session(right) that shows the values mentioned above.
iOS doesn't work ❌
The iOS app doesn’t work as expected. In other words, when I execute
then:
results.classifications[0].categories
has a maximum of 5 values ([the expected value is 521)results.classifications[1].categories
is always empty (the expected value is 5)the image below show a screenshot of the app (left) and a debugging session(right) that shows the values mentioned above.
ps1: I'm using
TensorFlowLiteTaskAudio 0.4.3
ps2: the iOS app works fine when using the default yamnet model. You can see in the image below thatresults.classifications[0].categories
has a maximum of 521 values (as expected)Question
Is there a bug on the pod “TensorFlowLiteTaskAudio”? Or is there some special configuration that should be done on iOS when integrating a model generated by “TensorFlow Lite Model Maker”? Or should I change something when generating the model .tflite for iOS?
Steps to Reproduce
ps: I created a repository to help you reproducing this behavior. It shows:
This is the link: https://github.com/ghashi/tensor-flow-lite-task-audio-poc . If you want to use it:
model-generator/model_maker_audio_classification-toreport.ipynb
or just skip this stepios
)ios/AudioClassification/TFLite/AudioClassificationHelper.swift