gbif / portal16

GBIF.org website
https://www.gbif.org
Apache License 2.0
24 stars 15 forks source link

Question: is /api/wikidata/species/ part of the gbif api and/or is there and outline of the code that gets the taxon identifiers #1677

Closed therobyouknow closed 2 years ago

therobyouknow commented 2 years ago

From looking he taxon identifiers box near the end of a species page, e.g. https://www.gbif.org/species/2285092

image

...my colleagues helped me deduce that code to get those values in that starts from around this line number in this repo:

If I assemble a URL endpoint call based on that, https://www.gbif.org/api/wikidata/species/2285092?locale=en

I get:

{"wikidataId":"Q3793860","wikidataUrl":"https://www.wikidata.org/wiki/Q3793860","identifiers":[{"id":"82321","url":"https://www.itis.gov/servlet/SingleRpt/SingleRpt?search_topic=TSN&search_value=82321","label":{"language":"en","value":"ITIS TSN"},"description":{"language":"en","value":"identifier for a taxon in the Integrated Taxonomic Information System"}},{"id":"138026","url":"https://www.marinespecies.org/aphia.php?p=taxdetails&id=138026","label":{"language":"en","value":"WoRMS-ID for taxa"},"description":{"language":"en","value":"identifier in the World Register of Marine Species"}},{"id":"57766","url":"https://eol.org/pages/57766","label":{"language":"en","value":"Encyclopedia of Life ID"},"description":{"language":"en","value":"eol.org item reference number"}},{"id":"8178","url":"http://www.fossilworks.org/cgi-bin/bridge.pl?a=taxonInfo&taxon_no=8178","label":{"language":"en","value":"Fossilworks ID for this taxon"},"description":{"language":"en","value":"identifier for an animal, plant, or microorganism in the Fossilworks database"}},{"id":"2285092","url":"https://www.gbif.org/species/2285092","label":{"language":"en","value":"GBIF taxon ID"},"description":{"language":"en","value":"taxon identifier in GBIF"}},{"id":"8db6f5aa-c1e3-44a1-b418-41b03981828e","url":"http://www.nzor.org.nz/names/8db6f5aa-c1e3-44a1-b418-41b03981828e","label":{"language":"en","value":"New Zealand Organisms Register ID"},"description":{"language":"en","value":"identifier for a taxon names in the New Zealand Organisms Register"}},{"id":"235479","url":"https://www.ncbi.nlm.nih.gov/taxonomy/235479","label":{"language":"en","value":"NCBI taxonomy ID"},"description":{"language":"en","value":"identifer for a taxon in the Taxonomy Database by the National Center for Biotechnology Information"}},{"id":"246981","url":"https://www.inaturalist.org/taxa/246981","label":{"language":"en","value":"iNaturalist taxon ID"},"description":{"language":"en","value":"identifier in iNaturalist"}},{"id":"NHMSYS0021056622","url":"https://data.nbn.org.uk/Taxa/NHMSYS0021056622","label":{"language":"en","value":"NBN System Key"},"description":{"language":"en","value":"identifier of a taxon in the National Biodiversity Network (UK)"}},{"id":"1409341","url":"http://www.irmng.org/aphia.php?p=taxdetails&id=1409341","label":{"language":"en","value":"IRMNG ID"},"description":{"language":"en","value":"identifier of a scientific name, in the Interim Register of Marine and Nonmarine Genera (IRMNG) database"}},{"id":"/m/0fq2n6q","url":"https://g.co/kg/m/0fq2n6q","label":{"language":"en","value":"Freebase ID"},"description":{"language":"en","value":"identifier for a page in the Freebase database. Format: \"/m/0\" followed by 2 to 7 characters. For those starting with \"/g/\", use Google Knowledge Graph identifier (P2671)"}},{"id":"2208","url":"http://www.boldsystems.org/index.php/TaxBrowser_TaxonPage?taxid=2208","label":{"language":"en","value":"BOLD Systems taxon ID"},"description":{"language":"en","value":"identifier for a taxon in boldsystems.org"}},{"id":"109502","url":"https://artsdatabanken.no/Taxon/_/109502","label":{"language":"en","value":"NBIC taxon ID"},"description":{"language":"en","value":"identifier for a taxon in the Norwegian Biodiversity Information Centre database, a database on species and ecosystems in Norway"}},{"id":"CC6E8784-FFD9-FF9B-FF54-FA4AF9BAD20C","url":"http://treatment.plazi.org/id/CC6E8784-FFD9-FF9B-FF54-FA4AF9BAD20C","label":{"language":"en","value":"Plazi ID"},"description":{"language":"en","value":"identifier for a taxon treatment at Plazi.org"}},{"id":"Gadila","url":"https://biodiversity.org.au/afd/taxa/Gadila","label":{"language":"en","value":"Australian Faunal Directory ID"},"description":{"language":"en","value":"identifier for a taxon on the Australian Faunal Directory website"}}],"iucnIdentifier":"","iucnThreatStatus":""}

which when pretty printed in VSCODE

{
  "wikidataId": "Q3793860",
  "wikidataUrl": "https://www.wikidata.org/wiki/Q3793860",
  "identifiers": [
    {
      "id": "82321",
      "url": "https://www.itis.gov/servlet/SingleRpt/SingleRpt?search_topic=TSN&search_value=82321",
      "label": { "language": "en", "value": "ITIS TSN" },
      "description": {
        "language": "en",
        "value": "identifier for a taxon in the Integrated Taxonomic Information System"
      }
    },
    {
      "id": "138026",
      "url": "https://www.marinespecies.org/aphia.php?p=taxdetails&id=138026",
      "label": { "language": "en", "value": "WoRMS-ID for taxa" },
      "description": {
        "language": "en",
        "value": "identifier in the World Register of Marine Species"
      }
    },
    {
      "id": "57766",
      "url": "https://eol.org/pages/57766",
      "label": { "language": "en", "value": "Encyclopedia of Life ID" },
      "description": {
        "language": "en",
        "value": "eol.org item reference number"
      }
    },
    {
      "id": "8178",
      "url": "http://www.fossilworks.org/cgi-bin/bridge.pl?a=taxonInfo&taxon_no=8178",
      "label": { "language": "en", "value": "Fossilworks ID for this taxon" },
      "description": {
        "language": "en",
        "value": "identifier for an animal, plant, or microorganism in the Fossilworks database"
      }
    },
    {
      "id": "2285092",
      "url": "https://www.gbif.org/species/2285092",
      "label": { "language": "en", "value": "GBIF taxon ID" },
      "description": { "language": "en", "value": "taxon identifier in GBIF" }
    },
    {
      "id": "8db6f5aa-c1e3-44a1-b418-41b03981828e",
      "url": "http://www.nzor.org.nz/names/8db6f5aa-c1e3-44a1-b418-41b03981828e",
      "label": {
        "language": "en",
        "value": "New Zealand Organisms Register ID"
      },
      "description": {
        "language": "en",
        "value": "identifier for a taxon names in the New Zealand Organisms Register"
      }
    },
    {
      "id": "235479",
      "url": "https://www.ncbi.nlm.nih.gov/taxonomy/235479",
      "label": { "language": "en", "value": "NCBI taxonomy ID" },
      "description": {
        "language": "en",
        "value": "identifer for a taxon in the Taxonomy Database by the National Center for Biotechnology Information"
      }
    },
    {
      "id": "246981",
      "url": "https://www.inaturalist.org/taxa/246981",
      "label": { "language": "en", "value": "iNaturalist taxon ID" },
      "description": { "language": "en", "value": "identifier in iNaturalist" }
    },
    {
      "id": "NHMSYS0021056622",
      "url": "https://data.nbn.org.uk/Taxa/NHMSYS0021056622",
      "label": { "language": "en", "value": "NBN System Key" },
      "description": {
        "language": "en",
        "value": "identifier of a taxon in the National Biodiversity Network (UK)"
      }
    },
    {
      "id": "1409341",
      "url": "http://www.irmng.org/aphia.php?p=taxdetails&id=1409341",
      "label": { "language": "en", "value": "IRMNG ID" },
      "description": {
        "language": "en",
        "value": "identifier of a scientific name, in the Interim Register of Marine and Nonmarine Genera (IRMNG) database"
      }
    },
    {
      "id": "/m/0fq2n6q",
      "url": "https://g.co/kg/m/0fq2n6q",
      "label": { "language": "en", "value": "Freebase ID" },
      "description": {
        "language": "en",
        "value": "identifier for a page in the Freebase database. Format: \"/m/0\" followed by 2 to 7 characters. For those starting with \"/g/\", use Google Knowledge Graph identifier (P2671)"
      }
    },
    {
      "id": "2208",
      "url": "http://www.boldsystems.org/index.php/TaxBrowser_TaxonPage?taxid=2208",
      "label": { "language": "en", "value": "BOLD Systems taxon ID" },
      "description": {
        "language": "en",
        "value": "identifier for a taxon in boldsystems.org"
      }
    },
    {
      "id": "109502",
      "url": "https://artsdatabanken.no/Taxon/_/109502",
      "label": { "language": "en", "value": "NBIC taxon ID" },
      "description": {
        "language": "en",
        "value": "identifier for a taxon in the Norwegian Biodiversity Information Centre database, a database on species and ecosystems in Norway"
      }
    },
    {
      "id": "CC6E8784-FFD9-FF9B-FF54-FA4AF9BAD20C",
      "url": "http://treatment.plazi.org/id/CC6E8784-FFD9-FF9B-FF54-FA4AF9BAD20C",
      "label": { "language": "en", "value": "Plazi ID" },
      "description": {
        "language": "en",
        "value": "identifier for a taxon treatment at Plazi.org"
      }
    },
    {
      "id": "Gadila",
      "url": "https://biodiversity.org.au/afd/taxa/Gadila",
      "label": { "language": "en", "value": "Australian Faunal Directory ID" },
      "description": {
        "language": "en",
        "value": "identifier for a taxon on the Australian Faunal Directory website"
      }
    }
  ],
  "iucnIdentifier": "",
  "iucnThreatStatus": ""
}

The above steps resolve my question here:

https://discourse.gbif.org/t/given-a-gbif-human-readable-webpage-for-a-species-how-to-find-the-api-call-for-each-item-on-the-page/3134

But my further questions are: Is /api/wikidata/species/ part of the gbif API? Can it get records in bulk.

Thank you.

MortenHofft commented 2 years ago

Hi Rob is /api/wikidata/species/ part of the gbif api? No it is not a part of the public API. It is an internal unstable API that is used by the website. I wouldn't recommend using it. And it does not support doing it for records by the bulk.

is there and outline of the code that gets the taxon identifiers? I would ask you to use the Wikidata API. It is what we do. What is stopping you from using the approach I outlined in discourse? https://discourse.gbif.org/t/given-a-gbif-human-readable-webpage-for-a-species-how-to-find-the-api-call-for-each-item-on-the-page/3134/7

therobyouknow commented 2 years ago

Hi Morten thank you very much for your answers to my 2 questions that has helped me.

I will close this issue on that basis.

Background (only for information) I am looking into APIs that provide bulk return of data, to avoid the overhead of individual calls. So definitely the answer to say the API does not do bulk is good information - I know what it is meant for and I can then look into other different APIs. Also noted not to use this internal API - not a problem. Always interesting to learn about the code in any case.

Thanks again.

MortenHofft commented 2 years ago

My impression is that the Wikidata API is fairly slow and need considerable throttling if you are not to be blocked by the rate limiting.