mmindje / Data-management-FRC-RBIS

The projects under this repository are only for biodiversity data management.
0 stars 0 forks source link

Clean digitized fish data and add them to RBIS #47

Open mmindje opened 3 years ago

mmindje commented 3 years ago

Hi @Thacien1 and @DanielIgirimbabazi

we also need to verify the digitized fish data, clean them and add the new data on the RBIS

Thacien1 commented 3 years ago

Dear @helendallas,

Kindly have a look at the fish occurrence data file.

Thank you.

Final fish occurrence data file 20 July 2021_BDMT.xlsx

helendallas commented 3 years ago

@Thacien1 Thanks, I will tackle this tomorrow.

helendallas commented 3 years ago

@Thacien1 @mmindje Thanks, sorry for the delay in getting this to you. I noted several issues, highlighted here and in the attached file. Please work through these and correct, and ask at our session tomorrow if clarification is needed. Please also look at the documentation re preparing occurrence upload files so that these errors don't keep appearing.

Re Albertine Rift Conservation Society unpublished data -you need to check that this is already not included in GBIF

Final.fish.occurrence.data.file.20.July.2021_BDMT_HD.xlsx

Thacien1 commented 3 years ago

@Thacien1 @mmindje Thanks, sorry for the delay in getting this to you. I noted several issues, highlighted here and in the attached file. Please work through these and correct, and ask at our session tomorrow if clarification is needed. Please also look at the documentation re preparing occurrence upload files so that these errors don't keep appearing.

  • No Sampling Date given for any data - without a sampling date the data cannot be uploaded
  • Check highlighted genera in genus column and correct
  • Species includes Genus and species - this is wrong, it should only be species
  • Taxon - several taxa have double spaces - they wont upload if they don't match the master list exactly. Please check all of these and correct
  • Sampling methods - It si unusual to record fish sampling as visual observation as they need to be caught - either with a seine net, or fyke, or rod etc. Please check this. Also what is Acoustic recording - please can you elaborate. Similarly Stationary-point-count technique.
  • Collector/Owner - is always First name then Surname, - correct
  • Author(s) - remove extra spaces and check formatting
  • Year - several do not have the correct formatting, please correct
  • Source is the Source of the data if from database or thesis - it is left blank if a published report or article - correct
  • URL/DOI -you must never have both, DOI is preferable, if not available then URL - check and delete, also DOI must just be numbers, check and correct. Check with Dimas if you have issues linking to article.

Re Albertine Rift Conservation Society unpublished data -you need to check that this is already not included in GBIF

Final.fish.occurrence.data.file.20.July.2021_BDMT_HD.xlsx

Dear @helendallas,

Thank you so much for the comments. The team will work on it and get back to you as soon as possible.

Warm regards,

Thacien

Thacien1 commented 2 years ago

@Thacien1 @mmindje Thanks, sorry for the delay in getting this to you. I noted several issues, highlighted here and in the attached file. Please work through these and correct, and ask at our session tomorrow if clarification is needed. Please also look at the documentation re preparing occurrence upload files so that these errors don't keep appearing.

  • No Sampling Date given for any data - without a sampling date the data cannot be uploaded
  • Check highlighted genera in genus column and correct
  • Species includes Genus and species - this is wrong, it should only be species
  • Taxon - several taxa have double spaces - they wont upload if they don't match the master list exactly. Please check all of these and correct
  • Sampling methods - It si unusual to record fish sampling as visual observation as they need to be caught - either with a seine net, or fyke, or rod etc. Please check this. Also what is Acoustic recording - please can you elaborate. Similarly Stationary-point-count technique.
  • Collector/Owner - is always First name then Surname, - correct
  • Author(s) - remove extra spaces and check formatting
  • Year - several do not have the correct formatting, please correct
  • Source is the Source of the data if from database or thesis - it is left blank if a published report or article - correct
  • URL/DOI -you must never have both, DOI is preferable, if not available then URL - check and delete, also DOI must just be numbers, check and correct. Check with Dimas if you have issues linking to article.

Re Albertine Rift Conservation Society unpublished data -you need to check that this is already not included in GBIF

Final.fish.occurrence.data.file.20.July.2021_BDMT_HD.xlsx

Dear @helendallas ,

I hope that you are doing well!. The Team would like to share with you fish module updates. Please see the attached files. We tried to upload the data to the RBIS testing site but we failed. We tried it many times but the uploading process to both taxonomic data and occurrence data is loading unending up to now. We were thinking that maybe the issue is in the system features or our network issue.

Capture Final-fish-occurrence-data-file-BDT_24_November2021_final-to-be-submitted-without-ARCOS_BDMT-xlsx-Occurrence-data.xlsx RBIS-Fish-Master-List_24_November_2021_BDT-xlsx-Taxon-Master-List.xlsx

helendallas commented 2 years ago

Hi @Thacien1 @mmindje

Thanks for the fish master and data files.

Re Fish master file:

I successfully uploaded the master file but only 88 species. Your list has 90 species. I then downloaded the fish csv in taxon management and compared your file with this. You have two taxa that are included twice, although in one case it is a different species in the species column.

Here is the file with the taxa highlighted

RBIS-Fish-Master-List_24_November_2021_BDT-xlsx-Taxon-Master-List.xlsx

image

Re fish data:

Please clarify: Sampling methods - It si unusual to record fish sampling as visual observation as they need to be caught - either with a seine net, or fyke, or rod etc. Please check this. Also what is Acoustic recording - please can you elaborate. Similarly Stationary-point-count technique.

We have added functionality to add a url, when there is no DOI for scientific articles. Dimas will be rolling this to RBIS is January.

I uploaded fish data file

image

It was completed but there were some errors. 3 records were species not on the master list including: error message (last column in the attached file: Taxonomy does not exist for this group

Then there was an issue with the DOI for the other records. This may be resolved with the january update Error file Fish data upload.xlsx

Please can you check these.

mmindje commented 2 years ago

Dear @helendallas

We do not know if it is the internet on our side or an issue we cannot explain. We are testing the fish occurrence data upload but the loading cannot exceed 1/192. Actually, we added new data. The master list is quickly loading.

Final-fish-occurrence-data-file-BDT_14 December 2021_final-to-be-submitted-without-ARCOS_BDMT-xlsx-Occurrence-data.csv

RBIS-Fish-Master-List_14 December_2021_BDT-xlsx-Taxon-Master-List.csv