search

iobis / BBNJ

0 stars 0 forks source link

Make list of ABNJ datasets and institutions #1

Open wardappeltans opened 5 years ago

wardappeltans commented 5 years ago

in case of multiple institutions linked to a single dataset: dataset X | institute Y dataset X | institute Z

pieterprovoost commented 5 years ago 
data <- jsonlite::fromJSON("https://api.obis.org/dataset?areaid=1")$results
datasetlist <- list()
for (i in 1:nrow(data)) {
  i_names <- data[i,]$institutes[[1]]$name
  c_names <- data[i,]$contacts[[1]]$organization
  all_names <- unique(na.omit(c(i_names, c_names)))
  datasetlist[[i]] <- data.frame(id = data[i,]$id, title = data[i,]$title, records = data[i,]$records, institute = if (is.null(all_names)) NA else all_names, stringsAsFactors = FALSE)
}
datasets <- dplyr::bind_rows(datasetlist)
write.csv(datasets, "datasets.csv", row.names = FALSE)

datasets.csv.zip

pieterprovoost commented 5 years ago

To get type from OceanExpert:

https://www.oceanexpert.net/api/v1/institution/6860.json

1   Academic
2   Research
3   Government
4   NGO
5   Private non-profit
6   Private commercial
7   International / Intergovernmental
wardappeltans commented 5 years ago

dag @pieterprovoost voeg jij de types nog toe aan de CSV file? lukt dat vandaag?

pieterprovoost commented 5 years ago

Ok will do.

pieterprovoost commented 5 years ago

https://github.com/iobis/BBNJ/tree/master/datasets_institutes

wardappeltans commented 5 years ago

@pieterprovoost kan jij de metadata contact roles nog toevoegen per dataset en per institute? Owner Contact Creator Metadata Provider …

pieterprovoost commented 5 years ago

The API currently returns contacts (which includes the role) and institutes (which includes the OceanExpert ID needed to get the institution type), but the link between the two is not there. This needs to be fixed first.

{
  "total": 892,
  "results": [
    {
      "id": "4354345d-7faf-4376-b326-ffbc04b6b0cd",
      "url": "http://ipt.obis.org/nonode/resource?r=wod2009",
      "archive": "http://ipt.obis.org/nonode/archive.do?r=wod2009",
      "published": "2018-10-01T02:00:00.000Z",
      "updated": "2019-03-24T21:04:47.884Z",
      "title": "World Ocean Database 2009",
      "citation": "Baranova, O.K, T.D. O'Brien, T.P. Boyer and I.V. Smolyar (2009). Plankton data. Chapter 16 in Boyer, T. P., J. I. Antonov , O. K. Baranova, H. E. Garcia, D. R. Johnson, R. A. Locarnini, A. V. Mishonov, T. D. O'Brien, D. Seidov, I. V. Smolyar, M. M. Zweng, 2009. World Ocean Database 2009. S. Levitus, Ed., NOAA Atlas NESDIS 66, U.S. Gov. Printing Office, Wash., D.C., 216 pp., DVDs",
      "abstract": "Quantitative, qualitative and biomass Plankton measurements.",
      "intellectualrights": "This work is licensed under a Creative Commons Attribution (CC-BY) 4.0 License",
      "feed": {
        "id": "f6ecda39-1165-4b36-86d3-61ada8abd7bb",
        "url": "http://ipt.obis.org/nonode/rss.do"
      },
      "institutes": [
        {
          "name": "Intergovernmental Oceanographic Commission of UNESCO",
          "oceanexpert_id": 6860,
          "oceanexpert_parent_id": null
        }
      ],
      "contacts": [
        {
          "role": null,
          "type": "contact",
          "givenname": "Todd",
          "surname": "O'Brien",
          "organization": "National Oceanic and Atmospheric Administration; National Marine Fisheries Service; Office of Science and Technology; Marine Ecosystems Division",
          "position": null,
          "email": "todd.obrien@noaa.gov",
          "url": null,
          "type_display": "Contact"
        },
        {
          "role": null,
          "type": "creator",
          "givenname": null,
          "surname": "OBIS Secretariat",
          "organization": "Intergovernmental Oceanographic Commission of UNESCO",
          "position": null,
          "email": "info@iobis.org",
          "url": "http://www.iobis.org",
          "type_display": "Creator"
        },
        {
          "role": null,
          "type": "metadataProvider",
          "givenname": null,
          "surname": "OBIS Secretariat",
          "organization": "Intergovernmental Oceanographic Commission of UNESCO",
          "position": null,
          "email": "info@iobis.org",
          "url": "http://www.iobis.org",
          "type_display": "Metadata Provider"
        }
      ],
      "nodes": [
        {
          "id": "310922b4-9d0c-4de1-92d7-9b442d34765b",
          "name": "OBIS Secretariat"
        }
      ],
      "records": 627485
    },
    {
      "id": "d77428ad-ca28-4f5a-8ef5-d4a8f4053d65",
      "url": "http://ipt.vliz.be/eurobis/resource?r=sahfos-cpr-zoo",
      "archive": "http://ipt.vliz.be/eurobis/archive.do?r=sahfos-cpr-zoo",
      "published": "2019-04-02T20:28:09.000Z",
      "updated": "2019-04-03T16:32:17.194Z",
      "title": "Continuous Plankton Recorder (Zooplankton)",
      "citation": "Johns D (2011): Continuous Plankton Recorder Dataset (SAHFOS) - Zooplankton. v4.1. Sir Alister Hardy Foundation for Ocean Science (SAHFOS). https://doi.org/10.7487/2a0udi",
      "abstract": "SAHFOS is unique in having comparable data on the geographical distribution, seasonal cycles and year-to-year changes in abundance of plankton over a large spatial area. There is no other survey in the world with 80 years of marine plankton data.",
      "intellectualrights": "This work is licensed under a Creative Commons Attribution (CC-BY) 4.0 License",
      "feed": {
        "id": "e3dad797-a123-4e78-8473-5b0a295d3685",
        "url": "http://ipt.vliz.be/eurobis/rss.do"
      },
      "institutes": [
        {
          "name": "Sir Alister Hardy Foundation for Ocean Science",
          "oceanexpert_id": 8326,
          "oceanexpert_parent_id": null
        }
      ],
      "contacts": [
        {
          "role": null,
          "type": "metadataProvider",
          "givenname": "Derek",
          "surname": "Broughton",
          "organization": "The Sir Alister Hardy Foundation for Ocean Science (SAHFOS)",
          "position": null,
          "email": "derbro@sahfos.ac.uk",
          "url": null,
          "type_display": "Metadata Provider"
        },
        {
          "role": null,
          "type": "contact",
          "givenname": "David",
          "surname": "Johns",
          "organization": "The Sir Alister Hardy Foundation for Ocean Science (SAHFOS)",
          "position": null,
          "email": "djoh@mail.pml.ac.uk",
          "url": null,
          "type_display": "Contact"
        },
        {
          "role": null,
          "type": "creator",
          "givenname": "David",
          "surname": "Johns",
          "organization": "The Sir Alister Hardy Foundation for Ocean Science (SAHFOS)",
          "position": null,
          "email": "djoh@mail.pml.ac.uk",
          "url": null,
          "type_display": "Creator"
        },
        {
          "role": null,
          "type": "creator",
          "givenname": null,
          "surname": null,
          "organization": "The Sir Alister Hardy Foundation for Ocean Science The Laboratory (SAHFOS)",
          "position": null,
          "email": null,
          "url": null,
          "type_display": "Creator"
        }
      ],
      "nodes": [
        {
          "id": "4bf79a01-65a9-4db6-b37b-18434f26ddfc",
          "name": "EurOBIS"
        }
      ],
      "records": 454659
    },
    {
pieterprovoost commented 5 years ago

OceanExpert ID issue fixed in https://github.com/iobis/elastic-api/commit/c81c80560b8eb928f08d7c02a413f0f6322581b7

pieterprovoost commented 5 years ago

The institutions list now has OceanExpert type and contact role, but to be complete 265 institutions would need to be matched. See CSV files at https://github.com/iobis/BBNJ/tree/master/datasets_institutes