Deal with absences?

[niconoe]

To keep thins simple for the first version, I suggest that the alert system only shows "present" occurrences: the occurrences that are clearly marked as absences at GBIF would be ignored when data is imported.

At a later stage, we could decide to actually import those absences and deal with them (show them differently on the map/table, make sure alerts are not sent...).

@damianooldoni / @timadriaens / @SoVDH : do you agree with that iterative approach.

If so, next steps for me are:

• define exactly how a GBIF occurrence can be detected as an absence (probably a combination of occurrenceStatus, individualCount, ...)
• Adapt the import code so those are actually filtered out (for now, they are shown as presences, which is misleading)

[timadriaens]

yes, that seems the best thing to do (indeed we might need them to visualize management results). Indeed, absence on gbif is defined as: individual count says numerical 0 AND occurrence status says #"absent" (and the gbif portal now has a filter on that).

[damianooldoni]

I agree, Nico! To start with, please just download presences. As said by @timadriaens, this is the default option GBIF provides (see filter on the left in occurrence search page).

But, I think we don't need individualCount to detect absences (now or in the future): to download presences we just need to add to the download query the condition occurrenceStatus = present. If in the future we want to

1. download absences as well: remove occurrenceStatus = present from query
2. detect absences from the downloaded data: apply filter occurrenceStatus == absent

[niconoe]

We've now implemented the simple solution to not load absences for now. For robustness: we're excluding absences from the GBIF download, but we also filter during the import loop (useful for manually downloaded DwCA)