ReptileDB (id 1008): test report

[yroskov]

@gdower used database dump 2020_12_v2 + classification from http://www.reptile-database.org/db-info/taxa.html

First version on DEV: https://data.dev.catalogueoflife.org/dataset/2144/classification

[yroskov]

CLASSIFICATION

• [x] Empty subfamilies in ord Squamata - superfam Scincomorpha - fam Scincidae: - FIXED Acontinae Ateuchosauridae Ristellidae

Children taxa from the website:

Acontinae: children Acontias, Typhlosaurus https://reptile-database.reptarium.cz/advanced_search?taxon=Acontinae&submit=Search - these 2 genera are in subfamily NotAssigned in DEV

Ateuchosauridae: 1 child Ateuchosaurus (2 spp) https://reptile-database.reptarium.cz/advanced_search?taxon=Ateuchosauridae&submit=Search - the genus & 2 spp are in family Scincinae in DEV

Ristellidae: children Lankascincus, Ristella https://reptile-database.reptarium.cz/advanced_search?taxon=Ristellidae&submit=Search - these 2 genera are in family Sphenomorphinae in DEV

[yroskov]

• [x] family NotAssigned in superfamily Gymnophthalmoidea. To yr: from where are all its genera? - FIXED On the website, the superfamily has only 3 families (no space for NotAssigned).

[gdower]

Fixed those issues--it was caused by atypical formatting in the classification field.

[yroskov]

• [x] family Shinisauridae in superfam Platynota in DEV - it should be in superfamily Shinisauroidea: DEV:

Website:

[yroskov]

• [ ] Families from unranked group Amphisbaenia go to superfamily NotAssigned - It's a best what we can do on our side

[yroskov]

• [ ] Empty subfamily Calabariinae in ord Squamata - superfam Booidea - fam Boidae. It's one child species Calabaria reinhardtii is in NotAssigned subfamily

Website says, 1 sp Calabaria reinhardtii. https://reptile-database.reptarium.cz/species?genus=Calabaria&species=reinhardtii&search_param=%28%28taxon%3D%27Calabariinae%27%29%29 The species is in subfamily NotAssigned in DEV: https://data.dev.catalogueoflife.org/dataset/2144/classification?taxonKey=Squamata-Booidea-Boidae-Calabaria%20reinhardtii%20%28Schlegel%2C%201851%29

[yroskov]

• [ ] genus Buhoma in NotAssigned family in superfam Colubroidea - fam Colubridae. No idea what is a right place for the genus - website does not give answer.

[yroskov]

• [ ] family Xenotyphlopidae misplaced in NotAssigned superfamily of Squomata order. Should be in Superfamily Typhlopoidea:

[yroskov]

• [ ] order Crocodylia: there are no separation of children between 2 subfamilies in Alligatoridae family on the website.

DEV:

Website: order Crocodylia - suborder Eusuchia Family Alligatoridae Subfamily Alligatorinae Subfamily Caimaninae

CoL solution: ignore subfamilies in family Alligatoridae via clearinghouse

[yroskov]

TESTS OF FINAL VERSION (2020-12) on prod: https://data.catalogueoflife.org/dataset/1008/classification

• [x] Metadata: updated. (ver 2020-12; Received by COL: 2021-05-03)

• [x] Sectors: 4 orders Crocodylia, Rhynchocephalia, Squamata, Testudines established

• [x] Imported 11440 spp vs 10233 in ac19 (+1207, hmm, might be)

[yroskov]

ISSUES

• [ ] Inconsistent Name, 5. Left as they are, with quotation marks.

• [x] Blacklisted Epithet, 19. Incorrectly parsed names, marked as infraspecific names. All blocked in CoL. https://data.catalogueoflife.org/catalogue/3/dataset/1008/workbench?facet=rank&facet=issue&facet=status&facet=nomStatus&facet=nameType&facet=field&facet=authorship&facet=authorshipYear&facet=extinct&facet=environment&facet=origin&issue=blacklisted%20epithet&limit=100&offset=0 Example:

• [x] Indetermined, 44 - All blocked in CoL. https://data.catalogueoflife.org/catalogue/3/dataset/1008/workbench?facet=rank&facet=issue&facet=status&facet=nomStatus&facet=nameType&facet=field&facet=authorship&facet=authorshipYear&facet=extinct&facet=environment&facet=origin&issue=indetermined&limit=100&offset=0 Examples:

• [ ] Unmatched Name Brackets, 355 - A set fixed, but not all

[yroskov]

TASKS

• [x] ACC-SYN (same acc; same auth) - all syns blocked in CoL (in every(?) species)

• [x] Split genus, 2 genera (Hypsilurus & Lankascincus) - resolved as 1st & 4th (with 1 sp) = prov. acc.

Resolved 2021-05-24

[yroskov]

ReptileDB ver 2020-12; Received by COL: 2021-05-03 - synced 2021-05-24 (with subsps as synonyms). Published in CoL 2021-06-10.

[yroskov]

ReptiveDB ver 2021-05; received by CoL 2021-05-23. Imported on prod 2021-06-23 (with accepted subsps).

• [x] Imported acc 11570 spp, acc 3141 subspp. (in the email: 11570 species and 2192 subspecies)
• [x] Metadata: ok
• [x] Sectors: 4 broken sectors. Re-matched successfully.

ISSUES (assessed)

• [ ] Inconsistent Authorship, 72: https://data.catalogueoflife.org/catalogue/3/dataset/1008/workbench?decisionMode=_NOT_NULL&facet=rank&facet=issue&facet=status&facet=nomStatus&facet=nameType&facet=field&facet=authorship&facet=authorshipYear&facet=extinct&facet=environment&facet=origin&issue=inconsistent%20authorship&limit=100&offset=0 @gdower, there are incorrectly parsed synonyms, such as Crotalus pallidus günther Boulenger, 1896: 581) (fide The name appears as Crotalus pallidus GÜNTHER 1895 (fide BOULENGER 1896: 581) on the website https://reptile-database.reptarium.cz/species?genus=Crotalus&species=triseriatus&search_param=%28%28search%3D%27Crotalus+pallidus%27%29%29. I blocked such names in CoL for the moment.

• [ ] Doubtful Name. @gdower, there are few synonyms such as Ameiva oculata d'orbigny Bibron, 1837 which marked by the system as infraspecific names: https://data.catalogueoflife.org/catalogue/3/dataset/1008/workbench?decisionMode=_NOT_NULL&facet=rank&facet=issue&facet=status&facet=nomStatus&facet=nameType&facet=field&facet=authorship&facet=authorshipYear&facet=extinct&facet=environment&facet=origin&issue=doubtful%20name&limit=100&offset=0 The name appears as Ameiva oculata D’ORBIGNY & BIBRON 1837 on the website https://reptile-database.reptarium.cz/species?genus=Teius&species=oculatus&search_param=%28%28search%3D%27Ameiva+caelestis%27%29%29 I have corrected binomial, authorstring and rank in CoL via Complex Decisions for the moment.

• [ ] Unmatched Name Brackets. @gdower, set of synonyms with comment "fide" in authorstrings have been parsed incorrectly: https://data.catalogueoflife.org/catalogue/3/dataset/1008/workbench?decisionMode=_NOT_NULL&facet=rank&facet=issue&facet=status&facet=nomStatus&facet=nameType&facet=field&facet=authorship&facet=authorshipYear&facet=extinct&facet=environment&facet=origin&issue=unmatched%20name%20brackets&limit=100&offset=0&reverse=false I blocked such names in CoL for the moment.

[yroskov]

TASKS 2021-06-23

• [x] Broken decisions, 6490. RematchAll = 799 remains; deleted.

• [x] ACC-SYN infra (diff acc, same auth) & ACC-SYN infra (diff acc, diff auth) - cleaning redundant subspecies in synonymy (= blocking synonyms)

Resolved 2021-06-24

Synced 2021-06-24

[yroskov]

ReptileDB in ac21: 6,681/0 spp (vs 11,440/0 in June 18). Loss of 4,759 spp in CoL [during sync?].

@gdower hypothesis: loss caused by the decisions "block species".

I do not see a reason for blocking those accepted species. There is no tool to delete a block of filtered decisions at https://data.catalogueoflife.org/catalogue/3/decision?limit=100&offset=0&subjectDatasetKey=1008.

@gdower has deleted decisions programmatically 2021-08-20.

[yroskov]

TASKS after deletion of decisions reptileDB + blocked + species:

Resolved:

Synced 2021-08-20.

[yroskov]

New version 2021-11. Dump received 2021-11-15.

The importer failed for reptiledb, getting stuck on something with distribution gazetteers. https://github.com/CatalogueOfLife/backend/issues/1064 = FIXED Re-imported on DEV with the distribution data removed: https://data.dev.catalogueoflife.org/dataset/2144/classification

Imported to prod 2021-11-16

• [x] Imported acc 11690 & 3138 subspp (vs acc 11570 spp, acc 3141 subspp in May; in the email 11690 species and 2198 subspecies)
• [x] Metadata: date & version updated
• [x] Classification: class (1), order (4), suborder, superfamily, family, subfamily, genus
• [x] Sectors: 4 orders, ok

[yroskov]

• [x] ISSUES assessed

• [ ] Doubtful Name, 89 https://data.catalogueoflife.org/catalogue/3/dataset/1008/workbench?facet=rank&facet=issue&facet=status&facet=nomStatus&facet=nameType&facet=field&facet=authorship&facet=authorshipYear&facet=extinct&facet=environment&facet=origin&issue=doubtful%20name&limit=100&offset=0 @gdower, in this report, there are misparsed names which interpreted like that (page from authorstring is moved to species epithet): Anolis boulengeri :242 O'Shaughnessy, 1881 - in original database Anolis boulengeri O'Shaughnessy, 1881: 242 Anolis turmalis :278 O'Shaughnessy, 1875 - in original database Anolis turmalis O'Shaughnessy, 1875: 278

can we do something about this case? (So far, I left names without decision)

• [ ] Unparsable Name, 2106; https://data.catalogueoflife.org/catalogue/3/dataset/1008/workbench?facet=rank&facet=issue&facet=status&facet=nomStatus&facet=nameType&facet=field&facet=authorship&facet=authorshipYear&facet=extinct&facet=environment&facet=origin&issue=unparsable%20name&limit=100&offset=0

• [ ] Some author's name (Müller, Günther) starts with low case letter and interpreted as subspecies epithet For example, Liolaemus pictus müller & Hellmich, 1939 Dipsas bubalina günther Smith, 1943)

Another example: https://data.catalogueoflife.org/catalogue/3/dataset/1008/workbench?facet=rank&facet=issue&facet=status&facet=nomStatus&facet=nameType&facet=field&facet=authorship&facet=authorshipYear&facet=extinct&facet=environment&facet=origin&limit=50&offset=0&q=Liolaemus%20bellii

@gdower, I guess crawling/parsing need our further attention.

[yroskov]

TASKS 2021-11-16

• [x] Broken decisions, 629; rematchAll = 204 left; deleted all

Resolved:

Synced 2021-11-16 (with remaining issues)

[yroskov]

Markus: some vernacular names have the language given at the front of the actual name e.g. (E) Freckled Monitor or (G) Trauerwaran (https://github.com/CatalogueOfLife/data/issues/352#event-5625694715)

https://data.catalogueoflife.org/dataset/1008/taxon/Reptilia-Squamata-Platynota-Varanidae-Varanus%20tristis-41ecd4c74f9a68152e708c6f4640ae6f

@gdower, can your convertor script resolve this?

[yroskov]

Spotted problems (preview 2022-10-12):

• [x] Alligatoridae family (order Crocodylia): there is no separation of children between 2 subfamilies (i.e. empty subfamilies, all genera in NotAssigned subfamily) = FIXED in the clb through blocking empty subfams

• [ ] More empty nodes (child genus dropped in NotAssigned): subfamily: Calabariinae

subfamily: Elapinae

subfamily: Leiolepidinae:

[yroskov]

Attention: two separate genera Macrelaps and Micrelaps in the family Atractaspididae.

Peter, 2022-01-14: Yes, that’s correct (and potentially confusing): two genera (within the same family) that are not even closely related.

[yroskov]

Revisiting "decisions" after fixes https://github.com/CatalogueOfLife/testing/issues/184#issuecomment-1055567444

TASKS 2022-03-04

Resolved 2022-03-04:

Synced 2022-03-04

[yroskov]

Due to changes in Vertebrata classification (https://github.com/CatalogueOfLife/testing/issues/186#issuecomment-1828674947): Synonym "Reptilia" should be added with 4 orders Rhynchocephalia, Squamata, Crocodylia & Testudines. Implemented 2024-02-05: these four orders established as new sectors from the resource id 279229, synced; ReptileDB children taxa re-established as sectors under orders of resource id 279229.

ReptileDB re-synced 2024-02-05

AS A RESULT of implementing the classification from the resource id 279229 "COL Checklist Chordata Higher Classification, ver 1.0 / 2023-11-27", rank Class is missing now for all reptiles.

[yroskov]

Doing cleaning of outdated decisions... 2024-02-07: 1,351 outdated decisions left in CoL, mainly in ReptileDB. The same subspecies is accepted name and synonym to its parent species.

Need: (1) delete all outdated decisions [x] & (2) re-do all tasks

Resolved 2024-02-07:

Re-synced 2024-02-07

[yroskov]

ReptileDB ver 2023-09 / 2023-09-30; dump received from Peter 2024-03-21, plus classification harvested from http://www.reptile-database.org/db-info/taxa.html; imported 2024-03-21

Tested on DEV https://github.com/CatalogueOfLife/testing/issues/134#issuecomment-2013245082

• [x] Imported: 12162 spp (vs 11690 Nov 2021)

• [x] Metadata: OK

• [ ] Classification:

• Empty subfamilies in the family Alligatoridae; all children genera placed out of subfamilies

• Genus Calabaria is out of subfamily Calabariinae

• Empty subfamily Elapinae; children genera placed out of subfamily

• [ ] Sectors: 4 orders, ok

• [ ] Extinct taxa: n/a

Metrics

ISSUES assessed 2024-03-22

TASKS

[yroskov]

Have look on Ablepharus (Morethia) anomalus (Adelaidensis) Peters, 1874 Ablepharus seydeli DE Witte, 1933 Agama (Eremioplanis) lessonae DE Filippi, 1865 Amphiesma andreae Ziegler & LE Khac Quyet, 2006 Amphiesma metusia Inger, Zhao, Shaffer & WU, 1990 Anolis boulengeri O'SHAUGHNESSY, 1881 Acontias (Evesia) smithi Deraniyagala, 1934

[yroskov]

There are species names with very long authorstrings, for example: Bothriechis khwargi Arteaga, Pyron, Batista, Vieira, Pelayo, Smith, Barrio-Amorós, Koch, Agne, Valencia, Bustamante, Harris & Guayasamin, 2024 (https://zoobank.org/NomenclaturalActs/8BFF5643-306F-435A-B074-668B41C43291) Bothriechis rahimi Arteaga, Pyron, Batista, Vieira, Pelayo, Smith, Barrio-Amorós, Koch, Agne, Valencia, Bustamante, Harris & Guayasamin, 2024 etc.

[yroskov]

ReptileDB ver 2024-03 / 2024-03-28; dump received from Peter 2024-03-30, plus classification harvested from http://www.reptile-database.org/db-info/taxa.html; imported 2024-04-01

• [x] Imported: 12162 spp (vs 12162 in previous iteration; 11690 in Nov 2021)
• [x] Metadata: version applied as "2024-03 / 2024-03-28" following this page http://www.reptile-database.org/db-info/news.html
• [ ] Classification: genera Lankascincus & Ristella are outside class Reptilia = both should be in subfam Ristellinae - fam Scincidae = sectors established 2024-04-01
• Empty subfamilies in the family Alligatoridae; all children genera placed out of subfamilies Alligatorinae & Caimaninae = subfamilies blocked (as in previous update)
• Genus Calabaria is out of subfamily Calabariinae = Move Calabaria from parent Boidae to parent Calabariinae = does not work - Request failed with status code 400 You cannot move a taxon or synonym which is part of sector 2036 = UNRESOLVED
• Empty subfamily Elapinae; children genera placed out of subfamily = UNRESOLVED
• [x] Sectors: OK; 34 sectors: 4 suborders, 15 superfamilies & 15 families
• [x] Extinct taxa: n/a

Metrics

ISSUES assessed 2024-04-01

TASKS

• Broken decisions, 4961 = deleted
• Outdated decisions, 58 = deleted

Redundant subspecies synonymy with parent species (unresolved): ACC-SYN infraspecies (different accepted, different authors) 2 of 218 ACC-SYN infraspecies (different accepted, same authors) 1 of 1860

Resolved 2024-04-01:

Synced 2024-04-01

[yroskov]

ReptileDB ver 2024-03 / 2024-03-28; new iteration imported 2024-04-02

• [x] Imported: 12162 spp (vs 12162 in previous iteration)

Classification:

• [x] Genera Lankascincus & Ristella are outside class Reptilia = FIXED

• [x] Empty subfamilies in the family Alligatoridae; all children genera placed out of subfamilies Alligatorinae & Caimaninae = All genera are in Alligatorinae (incl. genus Caiman, by mistake) and Caimaninae is empty = IGNORE Alligatorinae and BLOCK Caimaninae = applied 2024-04-03

• [x] Genus Calabaria is out of subfamily Calabariinae = FIXED

• [x] Empty subfamily Elapinae; children genera placed out of subfamily = BLOCK empty Elapinae = applied 2024-04-03

• [x] Sectors: all 36 sectors broken; missing subject: 2 genera deleted as sectors; 4 suborders, 15 superfamilies & 15 families = REPAIRED 2024-04-03

TASKS resolved 2024-04-03:

Synced 2024-04-03

[yroskov]

• [x] Missing brackets in some subsequent combinations https://github.com/CatalogueOfLife/data/issues/667#event-13072129254 = FIXED 2024-06-07

[yroskov]

ReptileDB ver 2024-03 / 2024-03-28; new iteration imported 2024-06-07

• [x] Imported: 12162 spp (vs 12162 in previous iteration) Classification:
• [x] Empty subfamilies in the family Alligatoridae; all children genera placed out of subfamilies Alligatorinae & Caimaninae = All genera are in Alligatorinae (incl. genus Caiman, by mistake) and Caimaninae is empty = need to IGNORE Alligatorinae and BLOCK Caimaninae = decision is in place 2024-06-07
• [x] Empty subfamily Elapinae; children genera placed out of subfamily = need to BLOCK empty Elapinae = decision is in place 2024-06-07
• [x] Sectors: OK

TASKS

Resolved 2024-06-07:

Synced 2024-06-07