Systema Dipterorum (id 1101): test report

[yroskov]

Version 3.1 received 2021-05-12. Imported to prod: https://data.catalogueoflife.org/dataset/1101/about

(previous reports are in https://github.com/CatalogueOfLife/testing/issues/6)

• [x] Metadata: updated

• [x] Sector: order Diptera minus 4 families

  1. delete CIPA sectors
  2. re-establish and sync CCW (suprefamily Tipuloidea)
  3. establish SD sector as order Diptera
  4. block 4 families: Cylindrotomidae Limoniidae Pediciidae Tipulidae
  5. sync

As result, assembly tree looks like that:

[yroskov]

• [x] Subfamily names \nSarcophaginae & Sciop[hilnae https://github.com/CatalogueOfLife/data/issues/584

Both names blocked in CoL. Reported to Neal.

Systema Dipterorum re-synced 2023-11-13.

[yroskov]

• [x] Portion "\n" reported in subgenus name https://github.com/CatalogueOfLife/data/issues/588#issuecomment-1818908351 2 names in SD ver 4.2.2 of May 2023: subgenus \n = BLOCKED 2023-11-20 and species Rhypholophus (\n) simulans (Savchenko, 1973) = FIXED via Complex Decision 2023-11-20

https://www.checklistbank.org/catalogue/3/dataset/1101/workbench?facet=rank&facet=issue&facet=status&facet=nomStatus&facet=nameType&facet=field&facet=authorship&facet=authorshipYear&facet=extinct&facet=environment&facet=origin&limit=50&offset=0&q=%5Cn

Re-synced 2023-11-20

[yroskov]

• [x] Uninomials with prefixes F, T (e.g. FChironominae, FChloropinae, FTephritini, TLestremiinae) = blocked 2023-12-05

Systema Dipterorum 4.2.2, May 2023 re-synced 2023-12-05

After the check of PREVIEW 2023-12-06: F & T names were blocked as taxa, not names.

Test names: *FChironominae (next parent subfamily Chironominae) Kribiopelma albidum Kieffer, 1923 Kribiobius modestus Kieffer, 1923

Decision "Ignore" applied instead of "Block". Synced 2023-12-07. That's work: only names was blocked and children taxa synced in the CoL.

[yroskov]

• [ ] 204 pairs of duplicated subgenera https://www.checklistbank.org/catalogue/3/dataset/1101/duplicates?catalogueKey=3&category=uninomial&limit=500&minSize=2&mode=STRICT&offset=0&rank=subgenus&status=accepted&withDecision=false

Remains unresolved. Attempt to block subgenus as a rank: (1) vanished all subgenera from the tree & species names, (2) created "self-synonymy" (identical ACC-SYN). Blocking subgenus decision was reversed 2023-12-05. Duplicated subgenera are back (sic! PREVIEW 2023-12-07).

The list was sent to Neal 2023-12-05.

[yroskov]

Tests of Systema Dipterorum ver. 4.5, 2023-11-16 processed via TW by DD vs data by GO: https://github.com/CatalogueOfLife/testing/issues/244

[yroskov]

Systema Dipterorum ver. 5.0, 2024-01-08 processed via TW by DD; imported 2024-02-07

• [x] Imported: 177193 spp (vs 171,937 spp in 4.2.2, May 2023)
• [x] Metadata: Corrected Version / Issued 0.38.1 / 2024-02-07 --> 5.0, Jan 2024 / 2024-01-08 Added paragraph in Description: This version of the Systema Dipterorum data has been imported in TaxonWorks (the author of the import script is D. Dmitriev) and past soft validation there before exporting to the CoLDP format (the author of the export script is G. Ower).
• [ ] Classification: 17 subfamilies and many genera & species are outside families in the Tree root (inside order Diptera). Species are flagged as Prov. Acc. 7 species with the portion [GENUS NOT SPECIFIED] in the root.
• [x] Sectors: sector "Diptera" broken = FIXED 2024-02-12 CCW families Cylindrotomidae, Limoniidae, Pediciidae, Tipulidae should be blocked in Systema Dipterorum; plus subfamily Tipulinae (also, check genus Tipula in the root of Diptera) = all families re-blocked 2024-02-12
• [ ] Extinct taxa: missing extinct flag with species (0 spp now vs 3,792 in May 2023). (Field Epoch in the source spreadsheet should be used). There are extinct flags in all other ranks.
• [x] Genera with square brackets: 37 names with the portion [GENUS NOT SPECIFIED] (workbench, reverse ordering, 40 n/p) = blocked
• [x] Names start with F, T, \n, ? = none
• [x] Split subgenera, 7 = resolved

ISSUES assessed 2024-02-12

• Multi Word Monomial, 1: species name Muscinae ariciaeformes Bezzi & Stein, 1907 is interpreted as a synonym in a rank of subfamily (parent Diptera > Muscidae > Muscinae > Muscini) = blocked in CoL https://www.checklistbank.org/catalogue/3/dataset/1101/workbench?facet=rank&facet=issue&facet=status&facet=nomStatus&facet=nameType&facet=field&facet=authorship&facet=authorshipYear&facet=extinct&facet=environment&facet=origin&issue=multi%20word%20monomial&limit=100&offset=0

• Unparsable Authorship, 15: majority of cases are authorstrings with split name Gonz ález-Mora & Gonz ález = all 15 fixed in CoL https://www.checklistbank.org/catalogue/3/dataset/1101/workbench?facet=rank&facet=issue&facet=status&facet=nomStatus&facet=nameType&facet=field&facet=authorship&facet=authorshipYear&facet=extinct&facet=environment&facet=origin&issue=unparsable%20authorship&limit=100&offset=0

• Unusual Name Characters, 28: incomplete trinomial names like Sargus infuscatus var. [sic] Scopoli, 1763 (all synonyms) = blocked in CoL https://www.checklistbank.org/catalogue/3/dataset/1101/workbench?facet=rank&facet=issue&facet=status&facet=nomStatus&facet=nameType&facet=field&facet=authorship&facet=authorshipYear&facet=extinct&facet=environment&facet=origin&issue=parsed%20name%20differs&limit=100&offset=0

• Inconsistent Name, 840: incomplete trinomial names like Aedes subsp. holocinctus Edwards, 1941, Aedes var. var.hebrideus Edwards, 1926 and Amiota f. furcata Okada, 1960 (all synonyms) = blocked in CoL (var. var. portion, perhaps, for Dmitry's attention) https://www.checklistbank.org/catalogue/3/dataset/1101/workbench?facet=rank&facet=issue&facet=status&facet=nomStatus&facet=nameType&facet=field&facet=authorship&facet=authorshipYear&facet=extinct&facet=environment&facet=origin&issue=inconsistent%20name&limit=100&offset=400&reverse=false

• Rank Name Suffix Conflict, 2254: (for DD attention): https://www.checklistbank.org/catalogue/3/dataset/1101/workbench?facet=rank&facet=issue&facet=status&facet=nomStatus&facet=nameType&facet=field&facet=authorship&facet=authorshipYear&facet=extinct&facet=environment&facet=origin&issue=rank%20name%20suffix%20conflict&limit=100&offset=0

TASKS

• Broken decisions, 9601: deleted

• Outdated decisions, 373: deleted

• ACC-ACC species (different authors) 2 of 512: https://www.checklistbank.org/dataset/1101/duplicates?authorshipDifferent=true&category=binomial&limit=50&minSize=2&mode=STRICT&offset=0&status=accepted There are many cases like this: Hoplacephala excisa (Villeneuve, 1913) vs Hoplacephala excisa Villeneuve, 1913

Hoplacephala nigriventris (Villeneuve, 1913)
vs Hoplacephala nigriventris Villeneuve, 1913

Hoplacephala retroseta (Villeneuve, 1913)
vs Hoplacephala retroseta Villeneuve, 1913

Huttonobesseria verecunda (Hutton, 1901) vs Huttonobesseria verecunda Hutton, 1901

Hystricia cuestae (Engel, 1920) vs Hystricia cuestae Engel, 1920

Isomyia pseudolucilia (Malloch, 1928) vs Isomyia pseudolucilia Malloch, 1928

plus, few cases of two identical accepted species (full list): Empis (Polyblepharis) fedtschenkoi Shasmshev, 2023 Empis (Polyblepharis) hirsutitarsis Shamshev, 2023 Empis (Polyblepharis) sogdiensis Shamshev, 2023
Empis (Polyblepharis) sogdiensis Shasmshev, 2023 Holops anarayae Barahona-Segovia, 2021
Holops grezi Barahona-Segovia, 2021
Holops pullomen Baharona-Segovia, 2021
Paraclius brooksi Soares, Capellari & Ale-Rocha, 2023 Physoconops tentenvilu Baharona-Segovia, 2020 Polleniopsis bomdilaensis Bharti & Verves, 2016

ACC-ACC species (same authors) 0 of 342: https://www.checklistbank.org/dataset/1101/duplicates?authorshipDifferent=false&category=binomial&limit=50&minSize=2&mode=STRICT&offset=0&status=accepted Two identical accepted species: Agadasys hexablepharis Whittington, 2000 Amblypsilopus qinlingensis Yang & Saigusa, 2005
Amplisegmentum venezuelensis Winterton, 2021 etc.

Resolved 2024-02-12:

Synced 2024-02-12

[yroskov]

• [ ] 3.334 provisionally accepted taxa in Systema Dipterorum have a self-synonymizing synonym record: https://github.com/CatalogueOfLife/data/issues/627

TASKS does not detect such cases.

[mdoering]

They do: https://www.checklistbank.org/catalogue/3/dataset/1101/duplicates?acceptedDifferent=false&authorshipDifferent=false&catalogueKey=3&category=binomial&limit=50&minSize=2&mode=STRICT&rankDifferent=false&withDecision=false

[yroskov]

Systema Dipterorum ver. 5.0, 2024-01-08 processed via TW by DD; second iteration (bring back extinct spp); imported 2024-03-07

• [x] Imported: 177,193 spp (vs 177,193 spp)
• [x] Metadata: Corrected Version / Issued 0.39.0 / 2024-03-07 --> 5.0, Jan 2024 / 2024-01-08
• [x] Classification: as above.
• [x] Sectors: sector "Diptera" broken (subject is missing) = REPAIRED 2024-03-08. No blocked CCW families Cylindrotomidae, Limoniidae, Pediciidae, Tipulidae in Systema Dipterorum; plus subfamily Tipulinae (also, check genus Tipula in the root of Diptera) = all families re-blocked 2024-03-08. Species with the portion [GENUS NOT SPECIFIED] in the root = BLOCKED.
• [x] Extinct taxa: 4742 spp = FIXED!
• [x] Genera with square brackets: 28 names with the portion [GENUS NOT SPECIFIED] (workbench, reverse ordering, 40 n/p) = blocked.
• [x] Names start with F, T, \n, ? = none
• [ ] Split taxa (see in TASKS)

METRICS

TASKS

• [ ] Broken decisions, 6765 = no actions (because I have no assurance that decisions/CLB operates correctly)

• [x] Identical subfamilies approx. 45. Split subfamilies. = RESOLVED: (1) alternatives without authorstring ignored; (2) if both tribes without authorstrings, item with less species ignored.

• [x] Identical tribe 0 of 37. Split tribes. = RESOLVED: (1) alternatives without authorstring ignored; (2) if both tribes without authorstrings, item with less species ignored. However, decisions are not shown in the interface (see screenshot below) - for attention of @thomasstjerne (https://www.checklistbank.org/catalogue/3/dataset/1101/duplicates?catalogueKey=3&category=uninomial&limit=50&minSize=2&mode=STRICT&offset=0&rank=tribe&status=accepted&withDecision=false)

• [x] Identical genus 0 of 62. Split genera. = RESOLVED: alternatives without authorstring ignored. However, decisions are not shown in the interface (see screenshot below) - for attention of @thomasstjerne

• [x] Identical subgenus 0 of 7. Spit subgenera, 7 = RESOLVED: blocked alternative without authorstring - they are empty.

• [x] ACC-ACC species (different authors) 2 of 512:

Seems, a bug resolved. On 2024-03-11, ACC-ACC species (different authors) 512 of 512:

• [ ] ACC-ACC species (same authors) 0 of 342. Sampled: identical data. Each records with bigger ID in the pair was blocked.

The same problem: interface does not show results ofdecision application. Neither in the report nor in the panel. https://www.checklistbank.org/catalogue/3/dataset/1101/duplicates?authorshipDifferent=false&catalogueKey=3&category=binomial&limit=500&minSize=2&mode=STRICT&offset=0&status=accepted&withDecision=false

See comments on bugs - stopper: https://github.com/CatalogueOfLife/backend/issues/1300#issuecomment-1989262479

https://github.com/CatalogueOfLife/backend/issues/1300#issuecomment-1989284347

https://github.com/CatalogueOfLife/backend/issues/1300#issuecomment-1991695306

Synced 2024-03-18, probably with sets of unresolved duplicates:

[yroskov]

Re-do TASKS after software fixes, 2024-03-22

• Broken decisions: 6765 = deleted
• ACC-ACC species (same authors), 98 of 342: names with higher IDs blocked

Resolved 2024-03-22:

Re-synced 2024-03-22

[yroskov]

Systema Dipterorum ver. 5.2 of 2024-05-15 (as 0.41.1 / 2024-05-23) processed via TW by GO; 1st iteration ; imported 2024-05-23

• [ ] Imported: 216,630 spp (vs 177,193 spp); Synonym Count: 1 = something wrong
• [ ] Metadata: ver. as "0.41.1 / 2024-05-23" need to corrected
• [ ] Classification: Species with the portion [GENUS NOT SPECIFIED] in the root
• [ ] Extinct taxa: no flag
• [ ] Genera with square brackets: species with the portion [GENUS NOT SPECIFIED] (workbench, reverse ordering, 40 n/p)
• [x] Names start with F, T, \n, ? = none
• [ ] Split taxa (see in TASKS)

METRICS

• [x] Incorrect authorstring with family Acartophthalmidae Enderlein, Lopes, Hardy, Czerny, Hennig, McAlpine, Munroe, Brauer, Bergenstamm, Fleming, Townsend, Zimin, Leach, Samouelle, Zetterstedt, Agassiz, Marschall, Griffini, Acloque, Arias, Brues, Melander, Lehrer, Hull, Alexander, Lahille, Rohdendorf, Verves, Aldrich, Griffiths, Guimarães, Dugdale, Rübsaamen, Hedicke, McAlpine, Mamaev, Robineau-Desvoidy, Macquart, Swainson, Griffith, Pidgeon, Harris, Rondani, Desmarest, Lioy, Rye, Barrett, Hendel, Coe, Latreille, Burmeister, Carpenter, Bowden, Becker, Brundin, Bigot, Mesnil, Ussatchov, Sack, Bezzi, Stein, Williston, Pleske, Newman, Grimshaw, Verrall, Young, Nowicki, Stuckenberg, Schnabl, Bernardi, Duda, Kloet, Hincks, Shannon, Crozy, Nartshuk, Bickel, Byers, Philip, Westwood, Baranov, Venturi, Schiner, Walker, Roback, Shewell, Winnertz, Malloch, Billberg, Hutton, Brodie, White, Stenhammar, Saether, Zimina, Herting, Glumac, Goffe, Egger, Pascoe, Townes, Frey, Beschovski, Jones, Albuquerque, Cogan, Wesché, Handlirsch, Aczél, Séguy, Wahlgren, Peck, Crampton, Cook, Hong, Cockerell, Wirth, Stone, Vujić, Curran, Kalugina, Zumpt, Blanchard, Drensky, Sturtevant, Bromley, Mani, Shiraki, Bertrand, Loew, Pandellé, Okada, Harris, Alcock, Yabar, Hall, Evenhuis, Edwards, Bhatia, Keilin, Morrison, Kertész, Shatalkin, Anonymous, Sharp, White, Wiedemann, Vimmer, Mathis, Adisoemarto, Wood, Wilcox, Papavero, Costa, Bode, Sabrosky, Arnaud, Smirnov, Nagatomi, Iwata, Daniels, Springer, Dziedzicki, Mallo, Roonwal, Tonnoir, Belkin, Fallén, Haliday, Coquillett, Slosson, Knowlton, Cutler, Hesse, Zaitzev, Theobald, Krivosheina, Ashe, Maa, Gressitt, Steyskal, Wheeler, Erichson, Presl, Bellardi, Berthold, Martin, Dahl, Bibby, Zilahi-Sebess, Grünberg, Brèthes, Curtis, Wesenberg-Lund, Miyatake, Anduze, Sedman, Weems, Vossbrinck, Friedman, Thompson, Nagler, Stanescu, Verb, Eysell, Austen, Rossi, Bau, Wing, Fluke, Holloway, Theodor, Papp, Weidner, Kieffer, Wu, Wenzel, Tokunaga, Grunin, Thomson, Kessel, Maggioncalda, Oken, Bequaert, Wingate, Carrera, Andretta, Woodley, Osten Sacken, Wulp, Stackelberg, Oldenberg, Colless, Riedel, Verbeke, Knutson, Lyneborg, Lameere, Lundström, Camras, Korneyev, Barnes, Heer, Jeannel, Elouard, Andersson, Blanchard, Boyes, Illingworth, Vaillant, Wandolleck, Kröber, Prado, Rafinesque, Perty, Doleschall, Philippi, Jaennicke, Hippa, Ass, Needham, Wasmann, Breddin, Börner, Wheeler, Seebold, Hardy, Meijere, Surcouf, Comstock, Hinton, Stephens, Meigen, Thon, Telford, Vockeroth, Roháček, Yang, Ren, Yang, Grichanov, Mazzarolo, Amorim, Liu, Chandler, Jaschhof, Ansorge, Norris, Richter, Lehr, Gaimari, Irwin, Labandeira, Schmitz, Gerstaecker, Hackman, Ovchinnikova, Barraclough, Freidberg, Tozoni, Andersen, Gagné, Kirby, Spence, Nitzsch, Neuhaus, Dyar, Borkent, Kano, Shinonaga, Kurahashi, Wang, Sasakawa, Marshall, Mostovski, Munari, Yeates, Lukashevich, Shcherbakov, Mason, Naglis, Brake, Kuznetzov, Szadziewski, Grootaert, Meuffels, Meunier, Shaw, Shaw, Greathead, Jaschhof, Didham, Pinto, Han, Schinz, Artigas, Wiegmann, Hancock, Krzemiński, Krzemińska, Saigusa, Nagatomi, Peris, González-Mora, Zloty, Sinclair, Pritchard, Séguy, Vilkamaa, Ševčík, Huang, Lin, Rindal, Guo, Wang, Michelsen, Zhang, Shih, Zhang, Krzeminska, Papier, Ebejer, Cherian, Shinimol, Zhu, Wang, Zhang, Fedotova, Sidorenko, Grimaldi, Cumming, Pape, Pimentel, Azar, Jaschhof, Li, Riccardi, Barták, Skartveit, Brown, Kung, Skibińska, Kaddumi, Pepinelli, Currie, Perkovsky, Lessard, Sidorenko, Lukashevich, Przhiboro, Plakidas, Tanasijtshuk, Oliveira, Ježek, Tang, Cranston, Rasnitsyn, Astakhov, Winterton, Ware, Hoffeins, Oldenburg, Mik, Wang, Blagoderov & Šifner, 2024 [1914]

Should be: Acartophthalmidae Czerny, 1928 = FIXED in 2nd iteration

• [x] Acartophthalmus coxatus Zetterstedt, 1848 = missing original combination as Agromyza coxata Zetterstedt, 1848 and missing brackets in the authorstring = FIXED in 2nd iteration

Both Acartophthalmidae & Acartophthalmus coxatus were correct in previous version.

[yroskov]

Systema Dipterorum ver. 5.2 of 2024-05-15 (as 0.41.1 / 2024-06-01) processed via TW by GO; 2nd iteration ; imported 2024-06-01

• [x] Imported: 176,894 (vs 216,630 spp in 1st iteration, vs 177,193 spp); "Synonym Count: 133,377" = OK (vs 135,855)

• [x] Metadata: ver. "0.41.1 / 2024-06-01" corrected as "5.2 of 2024-05-15" 2024-06-03

• [x] Classification: OK. Species with the portion [GENUS NOT SPECIFIED] in the root

• [x] Sector: order Diptera is broken (missing subject); RESTORED 2024-06-03. 4 CCW families Cylindrotomidae, Limoniidae, Pediciidae & Tipulidae = BLOCKED

• [x] Extinct taxa: OK (4737 spp vs 4742 spp)

• [x] Genera with square brackets: species with the portion [GENUS NOT SPECIFIED] (workbench, reverse ordering, 40 n/p) = 19 blocked

• [x] Names start with F, T, \n, ? = none

• [x] Split taxa (see in TASKS)

• [x] Family Acartophthalmidae Czerny, 1928 = OK

METRICS

ISSUES

TASKS

• Broken decisions, 5885 = all deleted
• Outdated decisions, 57 = no actions (ref: the problem of decisions between GSDs) = 0 after deletion of broken decisions

Resolved 2024-06-03:

Synced 2024-06-03

[yroskov]

Systema Dipterorum ver. 5.3 of 2024-07-17 (imported as 0.43.1 / 2024-08-12) processed via TW by GO; imported 2024-08-12

• [x] Imported: 177,314 (vs 176,894 spp); "Synonym Count: 133,479" = OK (vs 133,377)
• [x] Metadata: ver. "0.43.1 / 2024-08-12" corrected as "5.3 / 2024-07-17"
• [x] Classification: OK. Species with the portion [GENUS NOT SPECIFIED] in the root
• [x] Sector: Sector shown as healthy here https://www.checklistbank.org/catalogue/3/sector?limit=100&offset=0&subjectDatasetKey=1101, but with wrong subject as "Stenomyia fasciapennis Cresson, 1913" in expand view.
• [x] Sector expanded: incorrect subject = FIXED 2024-08-20

Assembly view:

• [x] Sector nested: 4 CCW families Cylindrotomidae, Limoniidae, Pediciidae & Tipulidae should be blocked in SD = RE-BLOCKED 2024-08-20

• [x] Extinct taxa: OK (4746 spp vs 4737 spp)

• [x] Genera with square brackets: species with the portion [GENUS NOT SPECIFIED] (workbench, reverse ordering, 40 n/p) = 22 blocked 2024-08-20

• [x] Names start with F, T, \n, ? = none

• [ ] Split taxa (see in TASKS)

• [x] Family Acartophthalmidae Czerny, 1928 = OK

METRICS

ISSUES assessed 2024-08-21

TASKS

• Broken decisions, 4980 = deleted

• Outdated decisions, 5 = no actions

• [x] ACC-ACC species (different authors) 2 of 61. Names reported to Neal: Bryophaenocladius pollexus Som, Mukherjee, Das & Charaborty, 2023 [incorrect spelling Chakraborty] = bloked
  Bryophaenocladius pollexus Som, Mukherjee, Das & Chakraborty, 2023 Gedanohelea liaoningensis Steber, Szadziewski & Wang, 2016 [incorrect spelling of Stebner] = bloked Gedanohelea liaoningensis Stebner, Szadziewski & Wang, 2016 Melanagromyza pseudolappae Gu, Fan & Sasakawa, 1991
  Melanagromyza pseudolappae Gu, 1991 Ophiomyia bispina Gu, 1991
  Ophiomyia bispina Gu, Fan & Sasakawa, 1991
  Phytomyza disjunctivena Gu, Fan & Sasakawa, 1991
  Phytomyza disjunctivena Gu, 1991 Urophora sevanensis Evsitgneev, 2023 [incorrect spelling of Evstigneev] = bloked Urophora sevanensis Evstigneev, 2023

• [x] Misspellings reported to Neal: Cdeeratopogonidae Chironlomidae Dolicvhopodidae Mycxetophilidae, Myceetophilidae (vs Mycetophilidae)

• [x] Identical family, genus, subgenus, etc. = pattern: "ignore" taxa with empty authorstring; ignore taxa with less spp; block taxa with 0 sp

Resolved 2024-08-21,22:

Synced 2024-08-22

[yroskov]

• [ ] Subgenus names in SD contain their accepted genus also as a synonym: https://github.com/CatalogueOfLife/data/issues/694#issuecomment-2315790762