Recognition Raw Scores returns [NaN, NaN, NaN, NaN]

[kyshr]

Hi @Caldarie I found a problem when I used GTM model. The Raw Score returns Nan with the latest version 0.2.1+1

D/AudioRecord(27796): stop(1446): 0x7543619a00, mActive:0 D/AudioRecord(27796): ~AudioRecord(1446): mStatus 0 D/AudioRecord(27796): stop(1446): 0x7543619a00, mActive:0 D/Tflite_audio(27796): Recording stopped. V/Tflite_audio(27796): Raw Scores: [NaN, NaN, NaN, NaN] D/Tflite_audio(27796): Recognition stopped. V/Tflite_audio(27796): result: {hasPermission=true, inferenceTime=89, recognitionResult=Background Noise} D/Tflite_audio(27796): Recognition Stream stopped

But I tried a non-GTM model and it works fine.

[Caldarie]

Hi @kyle-joseph

It very likely the audio file isn’t being fed properly into the model. Though, I’m unsure why as of this moment.

To help me pinpoint the problem:

1. can you provide the full logs?
2. Try the example gtm model from this repository and tell me if you receive the same error?
3. Tell me the values of your arguments from loadModel() and startAudioRecognition()?
4. Tell me whether you get the same error on both iOS and android? Or just one particular platform?

[kyshr]

Hello @Caldarie These are the parameters of the loadModel and startAudioRecognition(). I only used android platform.

TfliteAudio.loadModel(
  model: 'assets/model/coconut_acoustic_tm.tflite',
  label: 'assets/model/labels3.txt',
  inputType: 'rawAudio',
  numThreads: 1,
  isAsset: true,
);

}

result = TfliteAudio.startAudioRecognition( sampleRate: 44100, recordingLength: 44032, bufferSize: 22050, );

[kyshr]

V/Tflite_audio(17859): Raw Scores: [0.0022259208, 0.025381595, 0.6674544, 0.30493808] D/Tflite_audio(17859): Recognition stopped. V/Tflite_audio(17859): result: {hasPermission=true, inferenceTime=86, recognitionResult=Overmature} V/Tflite_audio(17859): Reached indicated number of inferences. D/AudioRecord(17859): stop(1523): 0x7568a85a00, mActive:1 V/Tflite_audio(17859): Recognition started. V/Tflite_audio(17859): Input shape: [1, 44032] V/Tflite_audio(17859): InputType: rawAudio D/AudioRecord(17859): stop(1523) done 2 D/AudioRecord(17859): stop(1523): 0x7568a85a00, mActive:0 D/AudioRecord(17859): ~AudioRecord(1523): mStatus 0 D/AudioRecord(17859): stop(1523): 0x7568a85a00, mActive:0 D/Tflite_audio(17859): Recording stopped. V/Tflite_audio(17859): Raw Scores: [NaN, NaN, NaN, NaN] D/Tflite_audio(17859): Recognition stopped. V/Tflite_audio(17859): result: {hasPermission=true, inferenceTime=75, recognitionResult=Background Noise}

After executing startAudioRecognition, it ran two extra executions that returns Nan even if the number of inference is the default 1. I can see the result in the first inference [0.0022259208, 0.025381595, 0.6674544, 0.30493808] but it returned Nan because of the two extra execution.

[Caldarie]

many thanks for the logs.

Ah, that is a very peculiar bug. Is this with android or iOS?

[kyshr]

You can view the full logs here.

https://docs.google.com/document/d/1hIVfPcQf0wxMbtMsWxO7sg8ykNX5afVO/edit?usp=sharing&ouid=114486676121174629562&rtpof=true&sd=true

I only used android @Caldarie . I don't know with iOS since I only use windows pc.

[Caldarie]

Understandable.

I will Investigate the issue and keep you updated

[kyshr]

Thank you so much @Caldarie since I really depend on your project for my Undergraduate Thesis. In my project I badly needed the raw scores to display the percentage of the inference.

[Caldarie]

Hi @kyle-joseph

I have found the offending code.

I will post an update soon.

[Caldarie]

hi @kyle-joseph

I have provided a fix in version 0.2.1+2. Please let me know if you still encounter the issue.

[kyshr]

Thank you so much @Caldarie