dataset metadata

[srand525]

We have two types of dataset metadata that we'd like to start populating into datasets.json

• metadata from the provider, e.g. geographical_coverage, temporal_coverage, etc.
• metadata generated from knowledge of what happens in a class - e.g. 'joins_to' (other datasets that the dataset can be merged with); or related_to (other datasets that the dataset may be a parent, child or subset of)

We can start to account for these in an original subdict, as we've done with RCPublications, like the below

      "id": "dataset-428",
      "provider": "National Science Foundation",
      "title":  "Higher Education Research and Development Survey",
      "alt_title": [
          "HERD",
          "Federal RePORTER"
      ],
      "url": "https://www.nsf.gov/statistics/srvyherd/",
      "description": "The survey collects information on R&D expenditures by field of research and source of funds and also gathers information on types of research, expenses, and headcounts of R&D personnel.",
      "original":{
          "joins_to":["dataset-493"]
      }
    }

But we should similarly decide what is the canonical set of metadata fields that we want to start incorporating into the subdict, and perhaps decide on standard field names here. @ceteri, do we want to include these joins_to or related_to fields, at this stage? This was prompted in part by our obtaining these documents from data providers, see below: https://github.com/NYU-CI/RCCustomers/blob/master/customers/NCSES/NCSES%20Database%20Diagram_With_Coleridge.pdf

[ceteri]

Good points, although the kinds of relations described here are directly part of KG representation. That comes later in the workflows.