pensoft / pensoft-interaction-tables

0 stars 0 forks source link

share example of how to read the review.tsv #7

Closed jhpoelen closed 3 years ago

jhpoelen commented 4 years ago

after completing #5 , GloBI review reports will be available . These review reports are generated using elton review and help to find potential issues with data processing. The current review reports consists of a 15 column table that detail typed review comments. The last column contains a json structured view into the review.

Here's an example of a structured report with some comments: (see also review.tsv.txt) :

reviewId | reviewDate | reviewer | namespace | reviewCommentType | reviewComment | archiveURI | referenceUrl | institutionCode | collectionCode | collectionId | catalogNumber | occurrenceId | sourceCitation | dataContext
-- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | --
6e186567-43dd-48d2-afe0-eb178cac0948 | 2020-07-13T17:58:09Z | GloBI automated reviewer (elton-0.3.5-SNAPSHOT) | local | note | source taxon name missing |   | http://openbiodiv.net/FB706B4E-BAC2-4432-AD28-48063E7753E4 |   |   |   |   |   |   | {"reviewId":"6e186567-43dd-48d2-afe0-eb178cac0948","reviewDate":"2020-07-13T17:58:09Z","reviewerName":"GloBI automated reviewer (elton-0.3.5-SNAPSHOT)","reviewCommentType":"note","reviewComment":"source taxon name missing","namespace":"local","context":{"Family Name":"Acanthaceae","Family Name_taxon_commonNames":null,"Family Name_taxon_externalId":"http://openbiodiv.net/4B689A17-2541-4F5F-A896-6F0C2EEA3FB4","Family Name_taxon_externalUrl":"http://openbiodiv.net/4B689A17-2541-4F5F-A896-6F0C2EEA3FB4","Family Name_taxon_id":"http://openbiodiv.net/4B689A17-2541-4F5F-A896-6F0C2EEA3FB4","Family Name_taxon_name":"Acanthaceae","Family Name_taxon_nameSource":null,"Family Name_taxon_nameSourceAccessedAt":null,"Family Name_taxon_nameSourceUrl":null,"Family Name_taxon_path":"Plantae \| Tracheophyta \| Magnoliopsida \| Lamiales \| Acanthaceae","Family Name_taxon_pathIds":null,"Family Name_taxon_pathNames":"kingdom \| phylum \| class \| order \| family","Family Name_taxon_rank":"family","Family Name_taxon_thumbnailUrl":null,"Host Plant":"Ruellia sp.","Host Plant_taxon_commonNames":null,"Host Plant_taxon_externalId":"http://openbiodiv.net/56F59D49-725E-4BF7-8A6D-1B1A7A721231","Host Plant_taxon_externalUrl":"http://openbiodiv.net/56F59D49-725E-4BF7-8A6D-1B1A7A721231","Host Plant_taxon_id":"http://openbiodiv.net/56F59D49-725E-4BF7-8A6D-1B1A7A721231","Host Plant_taxon_name":"Ruellia","Host Plant_taxon_nameSource":null,"Host Plant_taxon_nameSourceAccessedAt":null,"Host Plant_taxon_nameSourceUrl":null,"Host Plant_taxon_path":"Ruellia","Host Plant_taxon_pathIds":null,"Host Plant_taxon_pathNames":"genus","Host Plant_taxon_rank":"genus","Host Plant_taxon_thumbnailUrl":null,"Thrips species":"Copidothrips octarticulatus<br/> Thrips parvispinus","Thrips species_taxon_commonNames":null,"Thrips species_taxon_externalId":"http://openbiodiv.net/6A54156A-BE5C-44D7-A9E3-3902DA4CCFAC","Thrips species_taxon_externalUrl":"http://openbiodiv.net/6A54156A-BE5C-44D7-A9E3-3902DA4CCFAC","Thrips species_taxon_id":"http://openbiodiv.net/6A54156A-BE5C-44D7-A9E3-3902DA4CCFAC","Thrips species_taxon_name":"Copidothrips octarticulatus","Thrips species_taxon_nameSource":null,"Thrips species_taxon_nameSourceAccessedAt":null,"Thrips species_taxon_nameSourceUrl":null,"Thrips species_taxon_path":"Copidothrips octarticulatus","Thrips species_taxon_pathIds":null,"Thrips species_taxon_pathNames":"","Thrips species_taxon_rank":null,"Thrips species_taxon_thumbnailUrl":null,"referenceCitation":"Identification of the terebrantian thrips (Insecta, Thysanoptera) associated with cultivated plants in Java, Indonesia. http://openbiodiv.net/D37E8D1A-221B-FFA6-FFE7-4458FFA0FFC2. 10.3897/zookeys.306.5455","referenceDoi":"10.3897/zookeys.306.5455","referenceUrl":"http://openbiodiv.net/FB706B4E-BAC2-4432-AD28-48063E7753E4","studyTitle":"http://openbiodiv.net/FB706B4E-BAC2-4432-AD28-48063E7753E4","tableCaption":"Plants from which thrips have been collected in Java. <br/>"}}
7c779e26-146a-42a8-86eb-4f3e643bf5e6 | 2020-07-13T18:07:11Z | GloBI automated reviewer (elton-0.3.5-SNAPSHOT) | local | note | inconsistent column usage: found [6] data columns, but [9] column definitions |   | http://openbiodiv.net/A7353253-B642-44B4-A657-8C2086DBB472 |   |   |   |   |   |   | {"reviewId":"7c779e26-146a-42a8-86eb-4f3e643bf5e6","reviewDate":"2020-07-13T18:07:11Z","reviewerName":"GloBI automated reviewer (elton-0.3.5-SNAPSHOT)","reviewCommentType":"note","reviewComment":"inconsistent column usage: found [6] data columns, but [9] column definitions","namespace":"local","context":{"brachypterous":"Cardiospermum halicacabum","female":"","header-0":"Site 1","header-4":"","host":"","male":"","referenceCitation":"The soapberry bug, Jadera haematoloma (Insecta, Hemiptera, Rhopalidae): First Asian record, with a review of bionomics. http://openbiodiv.net/225D7F49-DF58-FFB7-8C62-FFA21923FFF5. 10.3897/zookeys.297.4695","referenceDoi":"10.3897/zookeys.297.4695","referenceUrl":"http://openbiodiv.net/A7353253-B642-44B4-A657-8C2086DBB472","studyTitle":"http://openbiodiv.net/A7353253-B642-44B4-A657-8C2086DBB472","tableCaption":"Collected individuals of Jadera haematoloma in the investigated sites of Kaohsiung City and Tainan City (for description of the sites see text).<br/>"}}
7c779e26-146a-42a8-86eb-4f3e643bf5e6 | 2020-07-13T18:07:14Z | GloBI automated reviewer (elton-0.3.5-SNAPSHOT) | local | note | inconsistent column usage: found [7] data columns, but [9] column definitions |   | http://openbiodiv.net/345BB624-A3F0-4F22-A7C0-120EC63E34FF |   |   |   |   |   |   | {"reviewId":"7c779e26-146a-42a8-86eb-4f3e643bf5e6","reviewDate":"2020-07-13T18:07:14Z","reviewerName":"GloBI automated reviewer (elton-0.3.5-SNAPSHOT)","reviewCommentType":"note","reviewComment":"inconsistent column usage: found [7] data columns, but [9] column definitions","namespace":"local","context":{"Cardiospermum halicacabum":"10.56±0.58<br/> (9.50–12.01) <br/> N = 47","Koelreuteria elegans subsp.formosana":"8.93±0.34<br/> (8.32–9.50) <br/> N = 49","body length (head–abdomen)":"10.64±0.55<br/> (9.37–11.88) <br/> N = 54","body length (head–wing)":"II-P (N = 3) <br/> III-A (N = 29) <br/> III-P (N = 21) <br/> IV-A (N = 1)","header-0":"males","labium":"9.02±0.40<br/> (8.05–9.90) <br/> N = 49","referenceCitation":"The soapberry bug, Jadera haematoloma (Insecta, Hemiptera, Rhopalidae): First Asian record, with a review of bionomics. http://openbiodiv.net/225D7F49-DF58-FFB7-8C62-FFA21923FFF5. 10.3897/zookeys.297.4695","referenceDoi":"10.3897/zookeys.297.4695","referenceUrl":"http://openbiodiv.net/345BB624-A3F0-4F22-A7C0-120EC63E34FF","studyTitle":"http://openbiodiv.net/345BB624-A3F0-4F22-A7C0-120EC63E34FF","tableCaption":"Body size (in mm) and relative length of the labium in specimens of different sex collected from different host plants (all macropterous).<br/>"}}

where the json part can be extracted using cat review.tsv | tail -n+3 | head -n1 | cut -f15 | jq . -

{
  "reviewId": "7c779e26-146a-42a8-86eb-4f3e643bf5e6",
  "reviewDate": "2020-07-13T18:07:11Z",
  "reviewerName": "GloBI automated reviewer (elton-0.3.5-SNAPSHOT)",
  "reviewCommentType": "note",
  "reviewComment": "inconsistent column usage: found [6] data columns, but [9] column definitions",
  "namespace": "local",
  "context": {
    "brachypterous": "Cardiospermum halicacabum",
    "female": "",
    "header-0": "Site 1",
    "header-4": "",
    "host": "",
    "male": "",
    "referenceCitation": "The soapberry bug, Jadera haematoloma (Insecta, Hemiptera, Rhopalidae): First Asian record, with a review of bionomics. http://openbiodiv.net/225D7F49-DF58-FFB7-8C62-FFA21923FFF5. 10.3897/zookeys.297.4695",
    "referenceDoi": "10.3897/zookeys.297.4695",
    "referenceUrl": "http://openbiodiv.net/A7353253-B642-44B4-A657-8C2086DBB472",
    "studyTitle": "http://openbiodiv.net/A7353253-B642-44B4-A657-8C2086DBB472",
    "tableCaption": "Collected individuals of Jadera haematoloma in the investigated sites of Kaohsiung City and Tainan City (for description of the sites see text).<br/>"
  }
}

@mdmtrv please let me know if you have any questions or improvement suggestions related to these review reports.

jhpoelen commented 3 years ago

Closing stale issue.