mmindje / Data-management-FRC-RBIS

The projects under this repository are only for biodiversity data management.
0 stars 0 forks source link

Odonates occurence data development #16

Closed mmindje closed 3 years ago

Thacien1 commented 4 years ago

Odonates data file is attached. Thank you. RBIS.Odonates.Data.26.July.2020._BDMT.xlsx

DanielIgirimbabazi commented 4 years ago

Odonates data file is attached. Thank you. RBIS.Odonates.Data.26.July.2020._BDMT.xlsx

Which is in progress*

helendallas commented 4 years ago

@helendallas @DanielIgirimbabazi Thank you for the Odonata data file. Both files were the same name and date. How is work going on the Odonata Master file. I noted it is alreeady added as a drop down lis in the data file, Please can you send me this as well. Looks like work is progressing nicley.

helendallas commented 4 years ago

@Thacien1 @mmindje PLease can you add the latest Odonata data file to this ticket, thanks.

Thacien1 commented 4 years ago

@Thacien1 @mmindje PLease can you add the latest Odonata data file to this ticket, thanks.

Thank you @helendallas . The odonates occurrence data file is now attached. RBIS.Odonates.Data.file.20.08.2020._BDMT.xlsx

helendallas commented 4 years ago

I have gone through the data file and made some revisions - in red. Points to note include:

My main question is in relation to the occurrence records falling in the Searching for Rwanda’s Flying Jewels: Agrion: Newsletter of the worldwide dragonfly assocation Volume 20(2): 100-104 publication. The dates of collections vary, as do the collectors etc., but if they are all from this publication then they need to be the same Author(s), Year, Source, Title, Reference category, and URL. Perhaps you have the original data from this study?

I have removed the explanatory header row and extra space row in preparation for the ingestion into RBIS.

Article attached:

Clausnitzer et al 2016 Searching fo Rwanda's flying jewels Agrion_Jul-2016.pdf

Here is the edited data file for you to use:

RBIS.Odonates.Data.file.20.08.2020._BDMT_HD.xlsx

helendallas commented 4 years ago

@mmindje @Thacien1 @DanielIgirimbabazi I have managed to get another 97 occurrence records of Odonata data via the OdonataMap, which is another JRS funded project that we are linked with. I have attached the excel file. The Source of this would be OdonataMap. You can add them to your data file.

OdonataMAP_Rwanda_for_Helen_Dallas.xlsx

Here is the ink to the website if you are interested ot have a look.
http://vmus.adu.org.za/vm_search.php?database=odonata&prj_acronym=OdonataMAP&db=odonata&URL=&Logo=images/odonata_logo.png&Headline=Odonata%20Atlas%20of%20Africa&Use_main_filter=0&User_id=&Full_name=&serve_sp_list=1&drop_down_list=&assessment=0&query_id=0&Vm_number=&recNo=0&numRows=0&start=0

Thacien1 commented 4 years ago

Excellent!!!. Thank you so much @helendallas . Lets work on them.

Thacien1 commented 4 years ago

Hello @helendallas , Below is an updated odonata occurrence data file. The team had run trough your comments but were confused on this comment "My main question is in relation to the occurrence records falling in the Searching for Rwanda’s Flying Jewels: Agrion: Newsletter of the worldwide dragonfly assocation Volume 20(2): 100-104 publication. The dates of collections vary, as do the collectors etc., but if they are all from this publication then they need to be the same Author(s), Year, Source, Title, Reference category, and URL. Perhaps you have the original data from this study?". We may clarify with you on it hopefully during the meeting.

RBIS.Odonates.Data.file.27.08.2020._BDMT.xlsx

helendallas commented 4 years ago

@mmindje @Thacien1 @DanielIgirimbabazi I have worked through the OdonataMap data and edited the data into the RBIS template. The blank records were "questionable"so I have deleted these as the identifications have not been confirmed by experts.

All data are species, except for one, Genus. Note that the Taxon Rank for this is noted as such. It is thus recorded in the appropriate column, and in the taxon column, which is the one actually ingested into the RBIS.

I added some wetland names where appropriate (you can add additional if appropriate) and edited the format of the collector name.

All data are "Database"with the source and title given in detail.

Please can you double check that all the species are already in the master list, and flag any that are not. Thanks

Here is the data file for adding to your RBIS Odonata file. OdonataMap data RBIS Format-Edited by HD.xlsx

helendallas commented 4 years ago

Hello @helendallas , Below is an updated odonata occurrence data file. The team had run trough your comments but were confused on this comment "My main question is in relation to the occurrence records falling in the Searching for Rwanda’s Flying Jewels: Agrion: Newsletter of the worldwide dragonfly assocation Volume 20(2): 100-104 publication. The dates of collections vary, as do the collectors etc., but if they are all from this publication then they need to be the same Author(s), Year, Source, Title, Reference category, and URL. Perhaps you have the original data from this study?". We may clarify with you on it hopefully during the meeting.

RBIS.Odonates.Data.file.27.08.2020._BDMT.xlsx

@Thacien1 Sorry I did not see this until our meeting had ended. Its mainly to confirm that all the occurrence records relate to the correct reference, even though the dates are sometimes earlier.

Thacien1 commented 4 years ago

New version of odonata occurrence data file.

RBIS.Odonates.Data.file_3.9.2020._BDMT.xlsx

helendallas commented 4 years ago

@Thacien1 Thanks Thacien. The odonate data file looks fine. So now you need to add the OdonataMap data to the RBIS Odonates file.

One question I have is if the Original Wetland Name should really rather be the Site description. In FBIS we had a River Name and a Site description, but perhaps this isnt clear for Rwanda.

Alternatively we could consider renaming: Wetland Name and description / or just Wetland Details

We will need to check with Dimas what the issues might be for compatability for RBIS and FBIS.

Thacien1 commented 4 years ago

@Thacien1 Thanks Thacien. The odonate data file looks fine. So now you need to add the OdonataMap data to the RBIS Odonates file.

One question I have is if the Original Wetland Name should really rather be the Site description. In FBIS we had a River Name and a Site description, but perhaps this isnt clear for Rwanda.

Alternatively we could consider renaming: Wetland Name and description / or just Wetland Details

We will need to check with Dimas what the issues might be for compatability for RBIS and FBIS.

Dear @helendallas , Thank you so much for this message. We are going to discuss on this and will get back to you with a team feedback tomorrow.

Thacien1 commented 4 years ago

Dear @helendallas ,

The team has analyzed your concern about wetland name issue and concluded that the field for Wetland name needs to be changed to Wetland details and the changes have been made in the occurrence data file accordingly. Thank you RBIS.Odonates.Data.file_4.9.2020._BDMT.xlsx

helendallas commented 3 years ago

@Thacien1 I am checking with Dimas, sure it will be fine. I suggest Wetland description.

Please confirm if you have integrate the OdonataMap data in the the Odonates data file.

Thacien1 commented 3 years ago

Thank you so much @helendallas for your suggestion. We agree with you that Wetland description fits well in that column. However, we haven’t yet integrated the OdonataMap data yet, we realized that they were quite inconsistent and we think you might be able to fetch them for us once again so that we can integrate them”

helendallas commented 3 years ago

@Thacien1 Please can you be more specific on the inconsistencies. Fetching them again wont make any difference especially since I edited the sheet quite considerably.

Thacien1 commented 3 years ago

Dear @helendallas , We are apologetic that we did not realized that you sent an updated OdonataMap data file. The team is going to work on this and will get back to you not later than tomorrow on Friday 5:00pm.

Thank you. Warm regards, @Thacien1

Thacien1 commented 3 years ago

@mmindje @Thacien1 @DanielIgirimbabazi I have worked through the OdonataMap data and edited the data into the RBIS template. The blank records were "questionable"so I have deleted these as the identifications have not been confirmed by experts.

All data are species, except for one, Genus. Note that the Taxon Rank for this is noted as such. It is thus recorded in the appropriate column, and in the taxon column, which is the one actually ingested into the RBIS.

I added some wetland names where appropriate (you can add additional if appropriate) and edited the format of the collector name.

All data are "Database"with the source and title given in detail.

Please can you double check that all the species are already in the master list, and flag any that are not. Thanks

Here is the data file for adding to your RBIS Odonata file. OdonataMap data RBIS Format-Edited by HD.xlsx

Hello @mmindje , @DanielIgirimbabazi , @Deborahcyu , consider this updated OdonataMap data file. Dear @helendallas , we will get back to you with complete odonata occurrence data file and odonata Master List file by tomorrow Friday at 5:00pm. Thank you.

mmindje commented 3 years ago

Okay Thacien, let us work on the files

Thacien1 commented 3 years ago

Dear @helendallas Kindly have a look on Odonata occurrence data file containing additional OdonataMap data

Thank you.

RBIS.Odonates.Data.file_11.Sept.2020._BDMT.xlsx

helendallas commented 3 years ago

@Thacien1 Please can you check why there are many records with no source? or authors or title. Where are these data from? There are only 3 datasets here right? As below, but many records have no source etc. Perhaps I am missing something?

Uyizeye E | 2020 | Antioch University PhD Thesis | Developing an Odonate-Based Index for Monitoring Freshwater Ecosystems in Rwanda: Towards Linking Policy to Practice through Integrated and Adaptive Management

Clausnitzer C, Kipping J, Dijkstra KDB | 2016 | Agrion: Newsletter of the worldwide dragonfly assocation Volume 20(2): 100-104 | Searching for Rwanda’s Flying Jewels

2020 | OdonataMap Virtual Museum, Animal Demography Unit, University of Cape Town

Please work with the attached file. RBIS Odonate Data file 2020_11_11.xlsx

helendallas commented 3 years ago

@Thacien1

I tried to ingest odonata file data into RBIS, and encountered a few errors. specifically: The attached file needs to be revised to include: Author(s) for rows in green, need to be added RBIS Odonate Data file 2020_11_11.xlsx

DanielIgirimbabazi commented 3 years ago

@Thacien1

I tried to ingest odonata file data into RBIS, and encountered a few errors. specifically: The attached file needs to be revised to include: Author(s) for rows in green, need to be added RBIS Odonate Data file 2020_11_11.xlsx

Dear @helendallas , We have been unable to retrieve the authorship for those missing information. We asked the data provider who also could not find the authors. However, we assume that the data are still raw and have not yet been published. This is why we left part of the authorship's field empty. We are looking forward to hearing your feedback on this issue. Thank you.

helendallas commented 3 years ago

@Thacien1 Who is the data provider? Is it their database? If so then they become the authors. Please provide details so that I can assist

Thacien1 commented 3 years ago

Thank you @helendallas

A data provider is Dr. Erasme Uyizeye, a recent PhD graduate from Antioch University.The data is not from his database, just a field collection data-set. It is basically raw data which has not yet been published.

helendallas commented 3 years ago

@Thacien1 Ok thanks. Then I will add him as author and note it as unpublished. Thanks I will sned the final master and data file once they has been successfully uploaded as thias is when the errors get picked up that I may have missed.

Thacien1 commented 3 years ago

@Thacien1 Ok thanks. Then I will add him as author and note it as unpublished. Thanks

Thank you so much @helendallas .

helendallas commented 3 years ago

I noticed some duplicates in the final check of the Odonate data, so ran a "concotenate" formulae and duplicate check. Please see the last column of the attached file. Please cna you check if these are indeed duplicates. If they are one should be deleted until no rows are highlighted in the last column. It is worth doing this for all data files so duplicate records are avoided.

Thanks - here is the file to work with. RBIS Odonate Data file 2020_11_11 For duplicate checking.xlsx

mmindje commented 3 years ago

I noticed some duplicates in the final check of the Odonate data, so ran a "concotenate" formulae and duplicate check. Please see the last column of the attached file. Please can you check if these are indeed duplicates? RBIS.Odonate.Data.file.2020_09_17.duplicate_checked_BDT.xlsx If they are one should be deleted until no rows are highlighted in the last column. It is worth doing this for all data files so duplicate records are avoided.

Thanks - here is the file to work with. RBIS Odonate Data file 2020_11_11 For duplicate checking.xlsx

Hi, @helendallas Thank you for the comment of the duplicate. We have worked on the file once again and here is the final one without the duplicate

mmindje commented 3 years ago

Dear @helendallas the attached is the new file after working on the duplicates RBIS.Odonate.Data.file.2020_09_17.duplicate_checked_BDT.xlsx

helendallas commented 3 years ago

Thanks team - final Odonate data file: RBIS Odonate Data file 2020_09_22 Final for RBIS V1.xlsx