Feat : add species whitelist

[alexbelgium]

https://github.com/Nachtzuster/BirdNET-Pi/issues/108

[alexbelgium]

Also fixes https://github.com/Nachtzuster/BirdNET-Pi/discussions/79 and https://github.com/Nachtzuster/BirdNET-Pi/discussions/51

[alexbelgium]

@dalogue1 : hi, I was thinking of an alternative - instead of adding a new instruction to the analysis logic we could rather just append the wanted species to the threshold file. Would this code be correct?

def predictSpeciesList(lat, lon, week):
    WHITELIST_LIST = None
    WHITELIST_LIST = loadCustomSpeciesList(os.path.expanduser("~/BirdNET-Pi/whitelist_species_list.txt"))
    l_filter = explore(lat, lon, week)

    for s in l_filter:
        if s[0] >= float(sf_thresh):
            # if there's a custom user-made include list, we only want to use the species in that
            if len(INCLUDE_LIST) == 0:
                PREDICTED_SPECIES_LIST.append(s[1])
    if len(WHITELIST_LIST) != 0:
        for species in WHITELIST_LIST:
            PREDICTED_SPECIES_LIST.append(species[0])

[Hiemstra87]

That would be great, I now have a specific list compiled of all Dutch birds I know have a reasonable chance of getting spotted. Would be better to have a dynamic list (on the occurrence list) + added specific species.

Does this account for species I add to the include list, that are already on the occurrence list? Or does that not even matter? (ie. does changes the detection parameters?)

[alexbelgium]

Hi, this works by adding the elements from the white-list to the detection list populated by threshold. So even if there is duplicates it isn't an issue.

[alexbelgium]

Hi @dalogue1, was it working with the updated code ?

[Nachtzuster]

Not too sure about the status: is this one is still being hashed out?

[alexbelgium]

Hi, well @dalogue1 was checking if the python is correct in terms of injecting the content of the whitelist in the detections list ; so indeed I'll put it back in draft until confirmation it works as intended thanks

[alexbelgium]

Just tested, and I can confirm that the content of the whitelist_species.txt appends to the detection_list in server.py => therefore the code serves its purposes of adding specific species beyond the normal list defined by threshold

[Nachtzuster]

This should refer to the 'Species Occurence Frequency Threshold' , right?

Assuming yes, we should probably advise here to first try the (I think) less strict v1 model. (side-note: I'm thinking to change the default back to the v1 model like the original BirdNET-Pi)

[Nachtzuster]

Full transparency: I'm not too keen on this PR: We know the model is less that perfect, but improvements should really be coming from 'upstream', trying to hack our way around any shortcomings might be opening us up to an unending game of whack-a-mole. I guess I'm saying we should be careful in adding these kind of features.

[alexbelgium]

I see your point, and remember a similar discussion on my proposal to convert species on-the-fly ; I do agree that bottom line it would be much better to improve the models but the key issue is the lack of control that we have on either the model training, or the probabilities of occurence from ebird...

In the end it is a bit the opposite of the "exclude" feature to removes false positives, this could be seen as an "include" for false negatives due to issues in the occurence probabilities...

I did indeed switched to the v1 model with better results ; however to be able to see a couple of very common owls I needed to lower the threshold to 0.001 (up from 0.00015 using the v2.4 model) ; which means I have to accept that I have quite exotic or improbable birds coming up (and a significant exclude list) ;-)

An alternative to merge the PR could be to create a simple "mod" feature, where the normal script executes custom instructions in a specific folder. For example, people who want noise-reduction, whitelist... could then place a script that includes this feature in a specific folder, and at each install (or update) the script executes. This methodology is inspired from how Linuxserver.io manages their images : they don't easily accept modification to their core code, but have set a folder (/custom-init.d for example) that allow users to add their own code

[Nachtzuster]

ok, this could give advanced users better results, understood