This PR enables continuous deployment of PUDL usage metrics. Main changes:
Created a daily github action called load-metrics that runs the run_data_update.py script.
Added some logic in respositories.py to collect all data source jobs. These jobs are organized into different repositories based on the database manager they use.
Create a new dagster resource for Postgres databases so jobs can load data to a Google Cloud SQL Postgres db.
Skip downstream ops if there are no datasette logs to process.
This PR enables continuous deployment of PUDL usage metrics. Main changes:
load-metricsthat runs therun_data_update.pyscript.respositories.pyto collect all data source jobs. These jobs are organized into different repositories based on the database manager they use.