Reducing false positives/ non divisible bufferRate outputs NaN

[bobosette]

Hi @Caldarie. I'm facing an issue reguarding the detection. I create my model with a lot of samples to recognize a certain noise, it works pretty well but tflite_audio recognizes also other noises like the one I would like to recognize. How can I fix this to adjust precision? Maybe I have to play with this parameters: detectionThreshold, averageWindowDuration, minimumTimeBetweenSamples, suppressionTime??

Thank you

[Caldarie]

Hi @bobosette,

If I am interpreting your issue correctly, you would like to output a specific noise while ignoring the others. Is that correct?

In that case, you could set the parameter outputRawScores as true. This should give you an array in string format "[0.2, 0.6, 0.2]". Each element represents how likely the model interprets a specific noise, for example [yes = 20%, no = 60%, left = 20%]

To solve your problem. you could just write an if statement to output the score once it reaches a certain threshold. To achieve this

1. Convert string value to array.
2. Get element of array. For example, result[1] where index of 1 represents "no"
3. If result[1] is greater than 0.5, print result.

Let me know if this solution answers your question.

[bobosette]

I already do this. My problem is with the false positive. But I can't understand if is a model problem or a tflite settings problem

[Caldarie]

In that case, increase your detectionThreshold to a higher threshold to reduce false positives. 0.7 is a good number, though you can push it to 0.8 if you have a really good model.

if you’re still are receiving false positives, then it’s highly likely the problem lies with your model.

[bobosette]

Ok thank you. And what if I change the buffersize (lower) to get more recognized time? Il Dom 2 Gen 2022, 15:57 Michael Nguyen @.***> ha scritto:

In that case, increase your detectionThreshold to a higher threshold to reduce false positives. 0.7 is a good number, though you can push it to 0.8 if you have a really good model.

if you’re still are receiving false positives, then it’s highly likely the problem lies with your model.

— Reply to this email directly, view it on GitHub https://github.com/Caldarie/flutter_tflite_audio/issues/28#issuecomment-1003728270, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADEWB4JTWQYWCWCVLU4J5XTUUBRURANCNFSM5LDQ3VRA . You are receiving this because you were mentioned.Message ID: @.***>

[Caldarie]

Lowering the bufferSize will increase recording time. However, as mentioned in the documentation, can reduce your model’s accuracy if lowered too much,

[bobosette]

I will do some tries and i will let u know

Il Dom 2 Gen 2022, 21:44 Michael Nguyen @.***> ha scritto:

Lowering the bufferSize will increase recording time. However, as mentioned in the documentation, can reduce your model’s accuracy if lowered too much,

— Reply to this email directly, view it on GitHub https://github.com/Caldarie/flutter_tflite_audio/issues/28#issuecomment-1003772385, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADEWB4K7OZNHSTFJXH4YB53UUC2JDANCNFSM5LDQ3VRA . You are receiving this because you were mentioned.Message ID: @.***>

[bobosette]

TfliteAudio.startAudioRecognition( sampleRate: 44100, recordingLength: 44032, bufferSize: 8000, numOfInferences: 99999, );

with 44032, if I put buffersize < 22016 the recognizer doesn't work,ì and I get this array of scores [NaN, NaN, NaN]. Now I will try to change detectionThreshold to 0.6 and averageWindowDuration to 1500

[Caldarie]

Hi, I will take a look at this possible bug.

In the mean time, can you try only changing detectionThreshold? averageWindowDuration may cause scores to output NaN.

[Caldarie]

Ah, yes you are right. There is a strange bug with lower values.

I will need time to investigate this.

[bobosette]

Best configuration I found at the moment is: detection threshold 0.7 averagewindowduration 1500 (never had NaN issue)

Il lun 3 gen 2022, 11:30 Michael Nguyen @.***> ha scritto:

Ah, yes you are right. There is a strange bug with lower values.

I will need time to investigate this.

— Reply to this email directly, view it on GitHub https://github.com/Caldarie/flutter_tflite_audio/issues/28#issuecomment-1003999859, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADEWB4PHQ6G62TQ6T7P3X53UUF3FTANCNFSM5LDQ3VRA . You are receiving this because you were mentioned.Message ID: @.***>

[Caldarie]

Hi, I have done some testing and have a general understanding to the issue.

1. The nan problem - it seems that if you use values that are not divisible to the recording length, i.e. 44032 you will get nan output. To fix this problem, use values that are divisible, for example: 44032, 22016, 11008, 5504... (If you absolutely want to use non divisible values, I will add a fix later, though it may take me some time to do so)

2. Inference accuracy - it seems that a buffer rate of values below 22016 will distort the accuracy of the model. I suggest that you reduce your sampleRate to values such as 22050 or 16000 to fix this distortion Note: that this will also slow your recording down further.

I hope my explanation helps you.

[bobosette]

Great. I tried with 11008 and I saw some improvements

Il Lun 3 Gen 2022, 14:09 Michael Nguyen @.***> ha scritto:

Hi, I have done some testing and have a general understanding to the issue.

1.

The nan problem - it seems that if you use values that are not divisible to the recording length, i.e. 44032 you will get nan output. To fix this problem, use values that are divisible, for example: 44032, 22016, 11008, 5504... 2.

Inference accuracy - it seems that a buffer rate of values below 22016 will distort the accuracy of the model. I suggest that you reduce your sampleRate to values such as 22050 or 16000. Note: that this will also slow your recording down further.

I hope my explanation helps you.

— Reply to this email directly, view it on GitHub https://github.com/Caldarie/flutter_tflite_audio/issues/28#issuecomment-1004080313, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADEWB4MQTBL5GWKOCPBUHHLUUGNZZANCNFSM5LDQ3VRA . You are receiving this because you were mentioned.Message ID: @.***>

[Caldarie]

Hi, good news. I have finally fixed the problem.

This update will be released with a planned feature, which features inferences on stored audio.

[bobosette]

Great

Il giorno mar 4 gen 2022 alle ore 04:13 Michael Nguyen < @.***> ha scritto:

Hi, good news. I have finally fixed the problem.

This update will be released on my new release, which features inferences on stored audio.

— Reply to this email directly, view it on GitHub https://github.com/Caldarie/flutter_tflite_audio/issues/28#issuecomment-1004497185, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADEWB4J4YE2UTQ3PO3FSKLLUUJQWZANCNFSM5LDQ3VRA . You are receiving this because you were mentioned.Message ID: @.***>