search

globalbioticinteractions / nomer

maps identifiers and names to other identifiers and names
GNU General Public License v3.0
19 stars 3 forks source link

allow to add providedTaxonRank, providedTaxonAuthorship and resolvedTaxonAuthorship, resolvedTaxonRank to [nomer list/append commands] #58

Closed jhpoelen closed 3 years ago

jhpoelen commented 3 years ago

as discussed with @seltmann

Desired:

providedTaxonNameId providedName providedAuthorship providedRank       resolvedAuthorship resolveRank          
https://www.discoverlife.org/mp/20q?search=Acamptopoeum+argentinum Acamptopoeum argentinum Doe, 2021 species HAS_ACCEPTED_NAME https://www.discoverlife.org/mp/20q?search=Acamptopoeum+argentinum Acamptopoeum argentinum Smith, 2021 species   Animalia | Arthropoda | Insecta | Hymenoptera | Andrenidae | Acamptopoeum argentinum https://www.discoverlife.org/mp/20q?search=Animalia | https://www.discoverlife.org/mp/20q?search=Arthropoda | https://www.discoverlife.org/mp/20q?search=Insecta | https://www.discoverlife.org/mp/20q?search=Hymenoptera | https://www.discoverlife.org/mp/20q?search=Andrenidae | https://www.discoverlife.org/mp/20q?search=Acamptopoeum+argentinum kingdom | phylum | class | order | family | species https://www.discoverlife.org/mp/20q?search=Acamptopoeum+argentinum
https://www.discoverlife.org/mp/20q?search=Acamptopoeum+calchaqui Acamptopoeum calchaqui Doe, 2021 species HAS_ACCEPTED_NAME https://www.discoverlife.org/mp/20q?search=Acamptopoeum+calchaqui Acamptopoeum calchaqui Smith, 2021 species   Animalia | Arthropoda | Insecta | Hymenoptera | Andrenidae | Acamptopoeum calchaqui https://www.discoverlife.org/mp/20q?search=Animalia | https://www.discoverlife.org/mp/20q?search=Arthropoda | https://www.discoverlife.org/mp/20q?search=Insecta | https://www.discoverlife.org/mp/20q?search=Hymenoptera | https://www.discoverlife.org/mp/20q?search=Andrenidae | https://www.discoverlife.org/mp/20q?search=Acamptopoeum+calchaqui kingdom | phylum | class | order | family | species https://www.discoverlife.org/mp/20q?search=Acamptopoeum+calchaqui
https://www.discoverlife.org/mp/20q?search=Acamptopoeum+colombiense Acamptopoeum colombiense Doe, 2021 species HAS_ACCEPTED_NAME https://www.discoverlife.org/mp/20q?search=Acamptopoeum+colombiense Acamptopoeum colombiense Smith, 2021 species   Animalia | Arthropoda | Insecta | Hymenoptera | Andrenidae | Acamptopoeum colombiense https://www.discoverlife.org/mp/20q?search=Animalia | https://www.discoverlife.org/mp/20q?search=Arthropoda | https://www.discoverlife.org/mp/20q?search=Insecta | https://www.discoverlife.org/mp/20q?search=Hymenoptera | https://www.discoverlife.org/mp/20q?search=Andrenidae | https://www.discoverlife.org/mp/20q?search=Acamptopoeum+colombiense kingdom | phylum | class | order | family | species https://www.discoverlife.org/mp/20q?search=Acamptopoeum+colombiense
https://www.discoverlife.org/mp/20q?search=Acamptopoeum+colombiensis Acamptopoeum colombiensis Doe, 2021 species SYNONYM_OF https://www.discoverlife.org/mp/20q?search=Acamptopoeum+colombiense Acamptopoeum colombiense Smith, 2021 species   Animalia | Arthropoda | Insecta | Hymenoptera | Andrenidae | Acamptopoeum colombiense https://www.discoverlife.org/mp/20q?search=Animalia | https://www.discoverlife.org/mp/20q?search=Arthropoda | https://www.discoverlife.org/mp/20q?search=Insecta | https://www.discoverlife.org/mp/20q?search=Hymenoptera | https://www.discoverlife.org/mp/20q?search=Andrenidae | https://www.discoverlife.org/mp/20q?search=Acamptopoeum+colombiense kingdom | phylum | class | order | family | species https://www.discoverlife.org/mp/20q?search=Acamptopoeum+colombiense
https://www.discoverlife.org/mp/20q?search=Acamptopoeum+fernandezi Acamptopoeum fernandezi Doe, 2021 species HAS_ACCEPTED_NAME https://www.discoverlife.org/mp/20q?search=Acamptopoeum+fernandezi Acamptopoeum fernandezi Smith, 2021 species   Animalia | Arthropoda | Insecta | Hymenoptera | Andrenidae | Acamptopoeum fernandezi https://www.discoverlife.org/mp/20q?search=Animalia | https://www.discoverlife.org/mp/20q?search=Arthropoda | https://www.discoverlife.org/mp/20q?search=Insecta | https://www.discoverlife.org/mp/20q?search=Hymenoptera | https://www.discoverlife.org/mp/20q?search=Andrenidae | https://www.discoverlife.org/mp/20q?search=Acamptopoeum+fernandezi kingdom | phylum | class | order | family | species https://www.discoverlife.org/mp/20q?search=Acamptopoeum+fernandezi
jhpoelen commented 3 years ago

With recent changes, I was able to produce the following output, using the (attached) properties file:

$ cat my.properties 
nomer.append.schema.output.example.taxon.rank.order=[{"column":0,"type":"path.order.id"},{"column": 1,"type":"path.order.name"},{"column": 2,"type":"path.order"}]
nomer.append.schema.output=[{"column":0,"type":"externalId"},{"column": 1,"type":"name"},{"column": 2,"type":"authorship"},{"column":3,"type":"rank"}]
nomer.schema.input=[{"column":0,"type":"externalId"},{"column": 1,"type":"name"},{"column": 2,"type":"authorship"},{"column": 3, "type":"rank"}]

my.properties.gz

and nomer:

$ nomer list --properties my.properties discoverlife | head
https://www.discoverlife.org/mp/20q?search=Acamptopoeum+argentinum  Acamptopoeum argentinum (Friese, 1906)  species HAS_ACCEPTED_NAME   https://www.discoverlife.org/mp/20q?search=Acamptopoeum+argentinum  Acamptopoeum argentinum (Friese, 1906)  species
https://www.discoverlife.org/mp/20q?search=Acamptopoeum+calchaqui   Acamptopoeum calchaqui  Compagnucci, 2004   species HAS_ACCEPTED_NAME   https://www.discoverlife.org/mp/20q?search=Acamptopoeum+calchaqui   Acamptopoeum calchaqui  Compagnucci, 2004   species
https://www.discoverlife.org/mp/20q?search=Acamptopoeum+colombiense Acamptopoeum colombiense    Shinn, 1965 species HAS_ACCEPTED_NAME   https://www.discoverlife.org/mp/20q?search=Acamptopoeum+colombiense Acamptopoeum colombiense    Shinn, 1965 species
https://www.discoverlife.org/mp/20q?search=Acamptopoeum+colombiensis    Acamptopoeum colombiensis   Shinn, 1965 species SYNONYM_OF  https://www.discoverlife.org/mp/20q?search=Acamptopoeum+colombiense Acamptopoeum colombiense    Shinn, 1965 species
https://www.discoverlife.org/mp/20q?search=Acamptopoeum+fernandezi  Acamptopoeum fernandezi Gonzalez, 2004  species HAS_ACCEPTED_NAME   https://www.discoverlife.org/mp/20q?search=Acamptopoeum+fernandezi  Acamptopoeum fernandezi Gonzalez, 2004  species
https://www.discoverlife.org/mp/20q?search=Acamptopoeum+inauratum   Acamptopoeum inauratum  (Cockerell, 1926)   species HAS_ACCEPTED_NAME   https://www.discoverlife.org/mp/20q?search=Acamptopoeum+inauratum   Acamptopoeum inauratum  (Cockerell, 1926)   species
https://www.discoverlife.org/mp/20q?search=Acamptopoeum+melanogaster    Acamptopoeum melanogaster   Compagnucci, 2004   species HAS_ACCEPTED_NAME   https://www.discoverlife.org/mp/20q?search=Acamptopoeum+melanogaster    Acamptopoeum melanogaster   Compagnucci, 2004   species
https://www.discoverlife.org/mp/20q?search=Acamptopoeum+nigritarse  Acamptopoeum nigritarse (Vachal, 1909)  species HAS_ACCEPTED_NAME   https://www.discoverlife.org/mp/20q?search=Acamptopoeum+nigritarse  Acamptopoeum nigritarse (Vachal, 1909)  species
https://www.discoverlife.org/mp/20q?search=Acamptopoeum+prinii  Acamptopoeum prinii (Holmberg, 1884)    species HAS_ACCEPTED_NAME   https://www.discoverlife.org/mp/20q?search=Acamptopoeum+prinii  Acamptopoeum prinii (Holmberg, 1884)    species
https://www.discoverlife.org/mp/20q?search=Acamptopoeum+submetallicum   Acamptopoeum submetallicum  (Spinola, 1851) species HAS_ACCEPTED_NAME   https://www.discoverlife.org/mp/20q?search=Acamptopoeum+submetallicum   Acamptopoeum submetallicum  (Spinola, 1851) species

In table form this would look something like:

https://www.discoverlife.org/mp/20q?search=Acamptopoeum+argentinum Acamptopoeum argentinum (Friese, 1906) species HAS_ACCEPTED_NAME https://www.discoverlife.org/mp/20q?search=Acamptopoeum+argentinum Acamptopoeum argentinum (Friese, 1906) species
https://www.discoverlife.org/mp/20q?search=Acamptopoeum+calchaqui Acamptopoeum calchaqui Compagnucci, 2004 species HAS_ACCEPTED_NAME https://www.discoverlife.org/mp/20q?search=Acamptopoeum+calchaqui Acamptopoeum calchaqui Compagnucci, 2004 species
https://www.discoverlife.org/mp/20q?search=Acamptopoeum+colombiense Acamptopoeum colombiense Shinn, 1965 species HAS_ACCEPTED_NAME https://www.discoverlife.org/mp/20q?search=Acamptopoeum+colombiense Acamptopoeum colombiense Shinn, 1965 species
https://www.discoverlife.org/mp/20q?search=Acamptopoeum+colombiensis Acamptopoeum colombiensis Shinn, 1965 species SYNONYM_OF https://www.discoverlife.org/mp/20q?search=Acamptopoeum+colombiense Acamptopoeum colombiense Shinn, 1965 species
https://www.discoverlife.org/mp/20q?search=Acamptopoeum+fernandezi Acamptopoeum fernandezi Gonzalez, 2004 species HAS_ACCEPTED_NAME https://www.discoverlife.org/mp/20q?search=Acamptopoeum+fernandezi Acamptopoeum fernandezi Gonzalez, 2004 species
https://www.discoverlife.org/mp/20q?search=Acamptopoeum+inauratum Acamptopoeum inauratum (Cockerell, 1926) species HAS_ACCEPTED_NAME https://www.discoverlife.org/mp/20q?search=Acamptopoeum+inauratum Acamptopoeum inauratum (Cockerell, 1926) species
https://www.discoverlife.org/mp/20q?search=Acamptopoeum+melanogaster Acamptopoeum melanogaster Compagnucci, 2004 species HAS_ACCEPTED_NAME https://www.discoverlife.org/mp/20q?search=Acamptopoeum+melanogaster Acamptopoeum melanogaster Compagnucci, 2004 species
https://www.discoverlife.org/mp/20q?search=Acamptopoeum+nigritarse Acamptopoeum nigritarse (Vachal, 1909) species HAS_ACCEPTED_NAME https://www.discoverlife.org/mp/20q?search=Acamptopoeum+nigritarse Acamptopoeum nigritarse (Vachal, 1909) species
https://www.discoverlife.org/mp/20q?search=Acamptopoeum+prinii Acamptopoeum prinii (Holmberg, 1884) species HAS_ACCEPTED_NAME https://www.discoverlife.org/mp/20q?search=Acamptopoeum+prinii Acamptopoeum prinii (Holmberg, 1884) species
https://www.discoverlife.org/mp/20q?search=Acamptopoeum+submetallicum Acamptopoeum submetallicum (Spinola, 1851) species HAS_ACCEPTED_NAME https://www.discoverlife.org/mp/20q?search=Acamptopoeum+submetallicum Acamptopoeum submetallicum (Spinola, 1851) species

Where the table is formatted in (left name) -[:relation] -> (right name) .

jhpoelen commented 3 years ago

See also attached discover.tsv.gz produced using:

$ nomer list --properties my.properties discoverlife | gzip > discoverlife.tsv.gz

discoverlife.tsv.gz

fyi @seltmann @egbot

(note that these nomer features are supported in upcoming release v0.2.8)

jhpoelen commented 3 years ago

Described functionality available in Nomer v0.2.8 . Please close this issue after review, or share review comments below.

seltmann commented 3 years ago

@jhpoelen looks good! There are some unexpected rows in the output. I checked a few of the taxa and they look like ones that are contain relationship HOMONYM_OF

{pathIds=null, externalUrl=https://www.discoverlife.org/mp/20q?search=Xylocopa+(Proxylocopa)+olivieri+alticola, nameSource=null, pathNames=null, nameSourceUrl=null, nameSourceAccessedAt=null, externalId=https://www.discoverlife.org/mp/20q?search=Xylocopa+(Proxylocopa)+olivieri+alticola, path=null, commonNames=null, authorship=Hedicke, 1938, name=Xylocopa (Proxylocopa) olivieri alticola, rank=subspecies, thumbnailUrl=null} |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- {pathIds=null, externalUrl=https://www.discoverlife.org/mp/20q?search=Xylocopa+(Zonohirsuta)+collaris, nameSource=null, pathNames=null, nameSourceUrl=null, nameSourceAccessedAt=null, externalId=https://www.discoverlife.org/mp/20q?search=Xylocopa+(Zonohirsuta)+collaris, path=null, commonNames=null, authorship=Maa, 1938, name=Xylocopa (Zonohirsuta) collaris, rank=species, thumbnailUrl=null} |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   {pathIds=null, externalUrl=https://www.discoverlife.org/mp/20q?search=Xylocopa+caffrariae+var+capensis, nameSource=null, pathNames=null, nameSourceUrl=null, nameSourceAccessedAt=null, externalId=https://www.discoverlife.org/mp/20q?search=Xylocopa+caffrariae+var+capensis, path=null, commonNames=null, authorship=Enderlein, 1903, name=Xylocopa caffrariae var capensis, rank=variety, thumbnailUrl=null} |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |  
seltmann commented 3 years ago

@jhpoelen a few homonyms look like they are still not being picked up. I am seeing _homonym in the provided:authorName and resolved:dwc:taxonName fields.

https://www.discoverlife.org/mp/20q?search=Andrena+(Taeniandrena)+gelriae+karelica | Andrena (Taeniandrena) gelriae karelica | f fuscata_homonym Niemelä, 1949 | subspecies | SYNONYM_OF | https://www.discoverlife.org/mp/20q?search=Andrena+gelriae | Andrena gelriae | van der Vecht, 1927 | species |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- https://www.discoverlife.org/mp/20q?search=Bombus+lapponicus+var+balcanicus | Bombus lapponicus var balcanicus | Friese, 1923 | variety | SYNONYM_OF | https://www.discoverlife.org/mp/20q?search=Bombus+pyrenaeus_homonym | Bombus pyrenaeus_homonym | Pérez, 1879 | species |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |   -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | --
jhpoelen commented 3 years ago

@seltmann thanks for sharing your notes on unexpected _homonym . Because these are not related to this particular issue (e.g., adding columns for authorship etc), I've opened a separate issue.

I am assuming that you were able to access the desired columns (e.g., providedTaxonRank, providedTaxonAuthorship) so I am closing this issue. Please feel free to re-open or comment if there's more work needed on this specific issue related to column definitions / views.

jhpoelen commented 3 years ago

oh, for related issue see https://github.com/globalbioticinteractions/nomer/issues/63 .

Thanks for being patient as I am trying to document and address these related, but separate, issues.