macaodha / batdetect2

Other
53 stars 10 forks source link

Batdetect2 detects Bats when there are no calls and not even audio features #27

Closed BEDOET closed 5 months ago

BEDOET commented 6 months ago

My files: Audiomoths have the option to generate a .wav file every X seconds. That can lead (combined with a frequency trigger) to "empty" files. Those are normally around 50kB big, have a length around 0.1 seconds but no audio features (at least not visible in a spectogram viewer like Audacity), since the Audiomoth didn't record anything in that timeperiod.

What I would expect: That batdetect2 detects 0 calls and classifies no species.

What is happening: Even though they are featureless, batdetect2 detects two calls. species, time, detection_prob, class_prob

species, time, detection_prob, class_prob
Myotis daubentonii, 0.0015, 0.12, 0.066
Myotis nattereri, 0.0825, 0.122, 0.106

Those results are consistent. I tested that with 25k different empty files.

When looking at the spectrogram on huggingface one vertical line is visible at 0.08s (that is detected as Mnat) while the Mdau bounding box is in the empty void. I don't know where that vertical line is coming from.

A subset of those empty files is in an otherwise empty repository.

I used batdetect2 version 1.0.8

mbsantiago commented 6 months ago

Hi @BEDOET,

Thanks for raising this issue and sharing those empty files. It seems our model is quite sensitive, even to subtle features in the spectrogram – although it should ideally assign a low probability to non-bat elements. We could potentially enhance this by training on a larger dataset of empty recordings, which might further lower the scores assigned to these features.

That said, the current probabilities are already fairly low, so filtering them out might be achievable by simply adjusting to a higher threshold.

Regarding the line you pointed out, I believe it's an artifact introduced during either the resampling or PCEN transformation. I'll definitely investigate this further to determine if we can safely remove it.

Cheers, Santiago

BEDOET commented 5 months ago

Hi Santiago, I dug a little bit deeper and it seems like the vertical line is not a resampling/ transformation but an Audiomoth artifact in the Wav-file. It is visible on Spectrograms created with the seeWave R-package but not Audacity which I used previously. So I guess you can close this issue in this regard.

However when using Audacitys function to silence those files to create this time real "empty" wav-files there are still species identified (det_prob <0.12, class_prob <0.11). So the issue of Batcall detection and classification (with low probablities) when there are no features persists. I uploaded the empty file (empty.wav) to the repository I previously linked.

But despite that batdetect2 is just awesome. It's fast, has a good classification and an even better detection of Myotis echolocation calls with 96 % detection accuracy in a sample dataset of mine.

Thanks

mbsantiago commented 5 months ago

Hi @BEDOET,

Thanks so much for delving deeper into this. We'll have your comments in mind about detections in "empty" wav files.

Cheers!