Closed yodakohl closed 5 years ago
False detections for Marvin if a very loud noisy signal is present. (https://soundcloud.com/user-837898227/marvin-false-detect)
I've been getting a lot of false detections with the marvin_sheila_small model, even with the sensitivity set up high, especially with something like TV on in the room.
I'm working on expanding the dataset with more background data. Did you mean low sensitivity? High sensitivity means it will detect more false positives.
I meant high, 0.9, have I this the wrong way around and 0.1 is what I want to be most accurate?
Just checked that I haven't published the marvin-sheila-small results yet. Here they are. If you set the Sensitivity to 0.1 will lead to less false positives, but will also be harder to trigger.
Accuracy clean
Sens: 0.1 Accuracy: 0.9085173501577287
Sens: 0.2 Accuracy: 0.9400630914826499
Sens: 0.3 Accuracy: 0.943217665615142
Sens: 0.4 Accuracy: 0.9779179810725552
Sens: 0.5 Accuracy: 0.9779179810725552
Sens: 0.6 Accuracy: 0.9810725552050473
Sens: 0.7 Accuracy: 0.9873817034700315
Sens: 0.8 Accuracy: 0.9873817034700315
Sens: 0.9 Accuracy: 0.9936908517350158
Sens: 0.95 Accuracy: 1.0
Sens: 0.99 Accuracy: 1.0
Accuracy noisy
Sens: 0.1 Accuracy: 0.8138801261829653
Sens: 0.2 Accuracy: 0.8517350157728707
Sens: 0.3 Accuracy: 0.8580441640378549
Sens: 0.4 Accuracy: 0.943217665615142
Sens: 0.5 Accuracy: 0.9305993690851735
Sens: 0.6 Accuracy: 0.9526813880126183
Sens: 0.7 Accuracy: 0.9337539432176656
Sens: 0.8 Accuracy: 0.9337539432176656
Sens: 0.9 Accuracy: 0.9558359621451105
Sens: 0.95 Accuracy: 0.9810725552050473
Sens: 0.99 Accuracy: 0.9810725552050473
False predictions per hour
Sens: 0.1 False Per Hour: 0.7794772727973933
Sens: 0.2 False Per Hour: 1.1692159091960899
Sens: 0.3 False Per Hour: 1.3640852273954382
Sens: 0.4 False Per Hour: 7.2101647733758885
Sens: 0.5 False Per Hour: 7.989642046173281
Sens: 0.6 False Per Hour: 8.574250000771327
Sens: 0.7 False Per Hour: 9.743465909967416
Sens: 0.8 False Per Hour: 10.133204546366112
Sens: 0.9 False Per Hour: 22.799710229323754
Sens: 0.95 False Per Hour: 37.60977841247423
Sens: 0.99 False Per Hour: 51.835238641026656
Some models may be prone to false detections when no signal is present. This can also trigger an initial false detection on startup. Will be fixed in the next model release.