glamod / glamod-ingest

Database preparation and ingestion for GLAMOD
BSD 2-Clause "Simplified" License
2 stars 1 forks source link

cdm-lens needs to provide a data policy and usage file for each request #13

Closed agstephens closed 3 years ago

agstephens commented 4 years ago
  1. Take response DataFrame, get the source_ids (before we remove from the DataFrame).

  2. Take each unique source_id and extract the following fields from the source_configuration table, e.g.:

SELECT product_name, product_references, product_citation FROM source_configuration WHERE source_id = 166;
  1. For each observation_id, take the first two characters as the country code.

  2. Look up the "Data Policy Link" from the country code.

  3. Compile that unique set into a text file and save it as part of the output zip file. Each item might include:

    • data policy link
    • product_name (i.e. the source name)
    • product_references
    • product_citation
agstephens commented 4 years ago

Format:

For each source:

Source: <source_id> - <product_name>
Product references:
....
Product citations:
...

Then, for each country code:

National Data Policy Information: (<abbrev>) - <country>
<link>
agstephens commented 4 years ago

Initial implementation is here:

https://github.com/glamod/cdm-lens/blob/master/cdm_interface/data_policies.py

Still requires:

  1. Loaded full.source_configuration table, or local version available as a file.
  2. Connection to the views.py and should be written to an output file.
agstephens commented 3 years ago

Linked to #21 and #39

agstephens commented 3 years ago

Implemented in commit: https://github.com/glamod/cdm-lens/commit/8b82e84528d1a086784ae92ce8c01b3c81bafa8d