Data download: all data for species X and Y in countries W and Z

[niconoe]

• Similar to what we do for RIPARIAS
• the approach seems sound, but currently we would only gain around 500 records
• An important feature of MICA is to distinguish observations VS catches. According to Emma, we could have an hardcoded list of cathes datasets, all the reste would be considered as an observation.
• Another think to check: is it okay to get all data for Belgium, Netherland and Germany, or should we filter per region? If so, can we do that filtering in the GBIF query, or should we implement our own filtering (against a shapefile?) in the import script?

(to be confirmed/rediscussed once with Emma before jumping to implementation)

[EmmaCartuyvels1]

Hi Nico, I think this is a really good idea! We should get all the data from Flanders, the Netherland and Germany, but an additional feature can be that we filter for observations in our project areas (I have send you those shapefiles, correct?) However I see that as another step that is not as urgent as implementing this

[niconoe]

Note to self: check how we can request data for Flanders (rather than Belgium) at GBIF.

If not easy/possible: alternative options are:

• load all data for Belgium
• filter out at import time in MICA

[EmmaCartuyvels1]

Hi Nico, Loading all data from Belgium would be fine as well!

[niconoe]

@EmmaCartuyvels1 : I'm in the process of implementing this! Could you give me the list of GBIF datasets that contains catches (occurrences found in those datasets will be considered as catches, all the others as observations)?

thanks

Note to self: afterwards, update GBIF_CATCHES_DATASET_KEY in settings and test it's taken into account at import time.

[EmmaCartuyvels1]

Hi Nico, These datasets contain only catches:

• MICA - Muskrat occurrences collected by VMM in Flanders, Belgium
• MICA - Muskrat occurrences collected by RATO in East Flanders, Belgium
• MICA - Muskrat and coypu occurrences collected by UVW in the Netherlands
• Mica - Muskrat, Raccoon and Coypu occurrences collected by ITAW in Germany
• Muskrat captures in Flanders, Belgium
• Muskrat trapping data in the Netherlands 1987 – 2014

However, in the future trapped and observed muskrats will be published together, at least for VMM. Maybe it's good to discuss this with @damianooldoni who is publishing the data

[damianooldoni]

Indeed, as rattenapp is collecting also data about IAS plants relevant for LIFE RIPARIAS we decided to take care of the mapping to DwC and publication of catches (relevant for LIFE MICA) AND observations/eradication (plants and animals).

The dataset mapping in repo riparias/vmm-rattenapp-occurrences is not far from being published (I am adding the catches of 2021 in a branch, almost done). In a second step (planned for June 2022) I would like to add the possibility to get an automated or semi automated workflow based on automatic export got via mail (VMM can set automatic mails with export it seems), GitHub Actions for running the DwC mapping and pywhip to allow some checks (via whip specs) which could trigger manual interventions if something new/suspect ocurrs in the DwC output.

[damianooldoni]

Title of this new dataset: "Rat control occurrences collected by VMM in Flanders, Belgium". The dataset "MICA - Muskrat occurrences collected by VMM in Flanders, Belgium" will be therefore not updated anymore as catches will be included in the new more general dataset.

[niconoe]

@damianooldoni: in the MICA dashboard, we need to clearly distinguish the catches from observations. The current logic is "by default, everything is an observation excepts if it comes from a specific list of "catches only" datasets - we have a hardcoded list for that.

As far as I understand, this logic won't work anymore in the future since the same dataset will contains both catches and observations. That's fine for me, we just need to find a clear way to discover them at the row level. Using the basisOfRecord field might be an elegant solution, if not possible we could do something like add "catches: true" to one t-of the *Remarks fields. What do you think?

[damianooldoni]

Yes @niconoe, I confirm what I briefly said during a coffie break, last Tuesday.

We are going to publish https://github.com/riparias/vmm-rattenapp-occurrences using samplingProtocol = "rat trap" exclusively for catches. For observations (plants or animals): samplingProtocol = "casual observation", while for eradication (plants) samplingProtocol = "eradication". See https://github.com/riparias/vmm-rattenapp-occurrences/blob/add-catches/sql/dwc_occurrence.sql#L15-L19

[niconoe]

That's now implemented with the new import script