Output metadata to accompany Datasette based on new Metadata structure

[katie-lamb]

We can generate table and column level annotations for the DB, which makes Datasette more useful and interesting.

Check out metadata.yml in devtools/datasette and Datasette documentation on doing this here

We are at the point when we can add column and table level metadata for the whole DB. See this issue as well.

To Do's:

• [x] Explore new metadata structure and familiarize where everything is
• [x] Decide which metadata features would enrich Datasette
• [x] Create to_yaml method
• [x] Figure out how the metadata data is best formatted into YAML
• [x] Create metadata.yml.jinja template that generates metadata.yml
• [x] Add in data source, table, and column level metadata
  • [x] Update template to iterate through resources and include table descriptions as well as label column field
  • [x] Figure out column level metadata + units
• [x] Call to_yaml from publish.sh and run publish.sh

Features to add with metadata

• [x] Database level: get metadata from DataSource - there is also metadata for the data sources listed within PUDL and FERC
  • DataSource name --> YAML name
  • title --> about
  • path --> about_url
  • license['title'] --> license
  • license['path'] --> license_url
  • could change description
• [x] Table level: get metadata from Resource
  • See this issue
  • description --> description
• [x] Column level: fields?
  • See this issue
  • Add units to columns (uses Pint)?
• [ ] Put facets in metadata? See this issue
  • documentation for facets

[zaneselvans]

Integrating units might be hard with nothing but the column names to indicate the units. We could also add another attribute to the Field data model that describes the units of the field if it's applicable. It would be great if that information could be integrated into the DB schema itself somehow too. But even just getting the metadata we've already got compiled exported and integrated with Datasette will be a big improvement!