SANParks Upload template

helendallas commented 4 months ago

Following discussion with SANParks, the upload template will need to be revised to accommodate additional fields. For now we only need to be able to upload these new fields, but they don't need to visualised or added to any filters, forms, etc.

Where possible we have used existing BIMS column headers, but there are extra ones needed.

Red are compulsory, black are optional, green shading indicates a dropdown list.

Some comments:

Including the Park or MPA name will allow checklist data without a latitude or longitude to be uploaded.
Date Accuracy specified if the sampling date is accurate or artificial
Verbatum Name (Scientific or common) is the name given by the collector of the data in the original dataset
Data Type specifies if the data are public, private or sensitive
Identified by specifies who identified the taxon (a name - keep same format as Collector)
Certainty of identification specifies the level of confidence
Accuracy of coordinates specifies the level of confidence if lat/long were provided.

Then in addition to this the SANParks team would like to be able to add extra fields for specific biodiversity modules. As discussed previously, these extra data don't need to be visualised but should be available through downloading a csv file.

Please see ticket #3688

Also for checklist type data, they will be added to the same template but will likely only have a subset of data added such as Park or MPA name, Sampling date, Date comment and Taxon

Supersedes ticket #3844

helendallas commented 4 months ago

Here is the xls file

SANParks Final Upload template_Occurrence data 2024_05_17.xlsx

helendallas commented 3 months ago

@dimasciput Please confirm when adding data at Park level, if a centroid will still be used or not to generate the geo reference. If former then data will need to be flagged, as not being a real coordinate.

dimasciput commented 3 months ago

@helendallas yes it will be a centroid, I will flag the sites.

dimasciput commented 3 months ago

@helendallas What is the difference between 'collector/owner' and 'identified by'? Will 'identified by' be a user object? And will it be queryable in the system as well?

helendallas commented 3 months ago

Comment from Dian:

A good example for this would be for herbarium voucher specimens.

A plant specimen will be collected by a particular person, but that person might not know what the name of the plant is. In this case the plant will be identified by a different person. Neither of these fields need to be queryable.

Another example is from iNaturalist (citizen science platforms) – different people will take photographs of plants, insects and other animals but they might not know what these are. The data that you get from that platform (which is also in GBIF) includes who took the photo (collector/owner) and who identified the plant or animal in the photo.

I hope this makes sense. If this information exists it can be used to determine how reliable the identification of something is. If something was identified by an expert we can be more confident that the identification is accurate.

dimasciput commented 3 months ago

@helendallas

You should now be able to upload the data to the system. However, you still need to add the latitude and longitude. If we have the park name, you can leave it empty. Here's an example of the CSV upload template:

test_sanparks.zip

helendallas commented 2 months ago

We were unable to test this because the tow fields were set as numerical instead of categorical. Field are :

Certainty of identification Accuracy of coordinates

Please update and then we can test again.

dimasciput commented 2 months ago

@helendallas I have updated the uploader, please test it again

helendallas commented 2 months ago

This isn't working.

I uploaded the Mammal Occurrence Upload template Attached here:

BIMS Mammal template 2024_07_29.xlsx

Then I tried to upload the occurrence data file for Mammals. Attached here:

BIMS MZNP 2023 Test data.xlsx

It does not get past record 2.

helendallas commented 2 months ago

Okay this seems to have worked. Data is being uploaded as shown in success file. Cannot check CVS download yet as many legacy FBIS columns included. CSV download needs to be a duplicate on the customised Mammal upload template please. So should draw on each biodiversity module upload template to determine column to include. This csv already has a ticket #3910

For now, closing this ticket until feedback from SANParks.

kartoza / django-bims

SANParks Upload template #3909