Discrepency between realtime/web reporting and the CLI?

[xconverge]

Something weird is going on, where I am able to save a result that makes no sense to me from the web UI, that has a pretty high confidence percentage, and then run it through the birdnet-go cli or BirdNET-Analyzer analyze.py script and get wildly different (much more correct) results than the web UI generated running realtime

If I run https://github.com/kahst/BirdNET-Analyzer/blob/main/analyze.py on these 4 files with the following files, I get the results I expect, 1 valid kestrel and 3 no-matches due to too low of confidence values.

4wavfiles.zip

python3 analyze.py --i ~/Downloads/inputs/ --o ~/Downloads/outputs/ --lat 37.7 --lon -122.4 --week 22 --sf_thresh 0.03 --sensitivity 1 --min_conf 0.7

I get 1 result which is correct, and 3 that are filtered out due to too low of a confidence (which is correct)

Selection   View    Channel Begin Time (s)  End Time (s)    Low Freq (Hz)   High Freq (Hz)  Common Name Species Code    Confidence  Begin Path  File Offset (s)
1   Spectrogram 1   1   0   3   0   15000   nocall  nocall  1.0 /Users/xxxxx/Downloads/inputs/actitis_macularius_79p_20240602T154933Z.wav   0

Selection   View    Channel Begin Time (s)  End Time (s)    Low Freq (Hz)   High Freq (Hz)  Common Name Species Code    Confidence  Begin Path  File Offset (s)
1   Spectrogram 1   1   0   3   0   15000   nocall  nocall  1.0 /Users/xxxxx/Downloads/inputs/anas_crecca_84p_20240602T152712Z.wav  0

Selection   View    Channel Begin Time (s)  End Time (s)    Low Freq (Hz)   High Freq (Hz)  Common Name Species Code    Confidence  Begin Path  File Offset (s)
1   Spectrogram 1   1   0   3.0 0   15000   American Kestrel    amekes  0.9354  /Users/xxxxx/Downloads/inputs/falco_sparverius_89p_20240602T160106Z.wav 0

Selection   View    Channel Begin Time (s)  End Time (s)    Low Freq (Hz)   High Freq (Hz)  Common Name Species Code    Confidence  Begin Path  File Offset (s)
1   Spectrogram 1   1   0   3   0   15000   nocall  nocall  1.0 /Users/xxxxx/Downloads/inputs/numenius_phaeopus_81p_20240602T160200Z.wav    0

If I then run birdnet-go CLI on the same files, with what I think are the same parameters in my config, I get the same results

root@88a412241874:/data/inputs# /bin/birdnet-go directory /data/inputs
BirdNET-Go build date: 2024-06-02T17:03:57Z, using config file: /root/.config/birdnet-go/config.yaml
BirdNET-Go build date: 2024-06-02T17:03:57Z, using config file: /root/.config/birdnet-go/config.yaml
Analyzing file: /data/inputs/actitis_macularius_79p_20240602T154933Z.wav
BirdNET GLOBAL 6K V2.4 FP32 model initialized, using 4 threads of available 4 CPUs
Analysis completed in 0 second(s)
Selection   View    Channel Begin File  Begin Time (s)  End Time (s)    Low Freq (Hz)   High Freq (Hz)  Species Code    Common Name Confidence
Analyzing file: /data/inputs/anas_crecca_84p_20240602T152712Z.wav
BirdNET GLOBAL 6K V2.4 FP32 model initialized, using 4 threads of available 4 CPUs
Analysis completed in 0 second(s)
Selection   View    Channel Begin File  Begin Time (s)  End Time (s)    Low Freq (Hz)   High Freq (Hz)  Species Code    Common Name Confidence
Analyzing file: /data/inputs/falco_sparverius_89p_20240602T160106Z.wav
BirdNET GLOBAL 6K V2.4 FP32 model initialized, using 4 threads of available 4 CPUs
Analysis completed in 0 second(s)
Selection   View    Channel Begin File  Begin Time (s)  End Time (s)    Low Freq (Hz)   High Freq (Hz)  Species Code    Common Name Confidence
1   Spectrogram 1   1   /data/inputs/falco_sparverius_89p_20240602T160106Z.wav  00:00:00    00:00:00    0   15000   amekes  American Kestrel    0.9354
Analyzing file: /data/inputs/numenius_phaeopus_81p_20240602T160200Z.wav
BirdNET GLOBAL 6K V2.4 FP32 model initialized, using 4 threads of available 4 CPUs
Analysis completed in 0 second(s)
Selection   View    Channel Begin File  Begin Time (s)  End Time (s)    Low Freq (Hz)   High Freq (Hz)  Species Code    Common Name Confidence

so then why did this show up for the actitis_macularius_79p_20240602T154933Z.wav file for example (it is where I copied/downloaded this audio from originally)

The good thing is, the CLI seems to be consistent and match what I saw with the BirdNET-Analyzer repo, but I have no idea why this detection shows up in the web UI and was detected with such a high confidence

Log from realtime birdnet-go at startup:

Starting analyzer in realtime mode. Threshold: 0.75, sensitivity: 1, interval: 15

My only hunch is a potential race condition in the realtime on the PendingDetections where it is getting a confidence from a different detection (of a valid/good bird that exceeds the threshold) and applying it which then allows it through/posts it

If these are both taking in pointers to the queue, I feel like there is a missing lock on the queue between the realtime processor and myaudio.ProcessData() or at the least these should be copies and not pointers to the actual queue that is being written to by myaudio.porcessData()?

func (p *Processor) processDetections(item *queue.Results) {

func (p *Processor) processResults(item *queue.Results) []Detections {

[tphakala]

It is likely due to realtime mode using 1.5 second overlap for analysis, so each 3 second chunk is analysed twice. Add 1.5 overlap for your CLI analyzer commands and see if you get matching results.

[tphakala]

Setting overlap to 1.5 does not produce high confidence detection of sand piper, but setting it to higher values like 2.8 and over push confidence over 75% threshold.

Detection accuracy seem to be very dependent on how bird call is aligned in analysed audio segment.

./birdnet-go file actitis_macularius_79p_20240602T154933Z.wav -t 0.75 -s 1 --overlap 2.99 --locale en
BirdNET-Go build date: 2024-06-02T16:37:51Z, using config file: /etc/birdnet-go/config.yaml
BirdNET-Go build date: 2024-06-02T16:37:51Z, using config file: /etc/birdnet-go/config.yaml
BirdNET GLOBAL 6K V2.4 FP32 model initialized, using 4 threads of available 4 CPUs
Analysis completed in 1 minute(s) 40 second(s)
Selection   View    Channel Begin File  Begin Time (s)  End Time (s)    Low Freq (Hz)   High Freq (Hz)  Species Code    Common Name Confidence
31  Spectrogram 1   1   actitis_macularius_79p_20240602T154933Z.wav 00:00:00    00:00:00    0   15000   sposan  Spotted Sandpiper   0.8506
2691    Spectrogram 1   1   actitis_macularius_79p_20240602T154933Z.wav 00:00:00    00:00:00    0   15000   comgre  Common Greenshank   0.8071
3131    Spectrogram 1   1   actitis_macularius_79p_20240602T154933Z.wav 00:00:00    00:00:00    0   15000   comgre  Common Greenshank   0.9202
3281    Spectrogram 1   1   actitis_macularius_79p_20240602T154933Z.wav 00:00:00    00:00:00    0   15000   comgre  Common Greenshank   0.7778
3321    Spectrogram 1   1   actitis_macularius_79p_20240602T154933Z.wav 00:00:00    00:00:00    0   15000   comgre  Common Greenshank   0.8388

[xconverge]

Quoting your message in the other spot to this issue to keep it straight:

"Latest BirdNET range filter model is likely to be defective, there is discussion about it here https://github.com/kahst/BirdNET-Analyzer/issues/296 "

[tphakala]

Range filter model is unrelated to confidence results, these are two separate issues.

[xconverge]

I see, so you are saying just the audio file alone through the model is generating a very high confidence. I assumed since I am seeing some birds that are not possible due to locale, the location and filtering due to location was relevant

[tphakala]

Range filter model is responsible of filtering out species based on location and date (week number), if you are seeing results for species which are not native to your region then problem is in range filtering, could be a threshold issue but also a defect in range filter model, latter is suspected with latest model. I will add option to switch back to previous range filter model.

Description of this issue is about difference in confidence levels during realtime detection and post analysis, that is related to "main" BirdNET AI model.

[xconverge]

I see, the 2nd example in this issue, "anas_crecca" (Definitely not native to my locale in the US) would be a better example for my confusion

I didn't necessarily mean to say or imply "confidence (only)" is different between the CLI and realtime specifically, just that the web UI returns different results than running it from the CLI (with the assumption that how location is used/applied/filtered played a role in this difference as well)

birdnet-go realtime/web gave it an 84% and showed it. Birdnet-go CLI did not return it as a result, I am unsure if this is due to confidence or the range filter.