coderxio / sagerx

Open drug data pipelines curated by pharmacists.

https://coderx.io/sagerx

Other

45 stars 12 forks source link

Airflow Logging into Postgres #251

Closed lprzychodzien closed 4 months ago

lprzychodzien commented 8 months ago

Problem Statement

Add airflow logging so we can track data pull occurrences and issues

Criteria for Success

Data available for easy dashboarding of data updates

jrlegrand commented 8 months ago

Should we just close https://github.com/coderxio/sagerx/issues/164 in favor of this issue you just created? I don't think the solution in #164 is necessarily exactly what we want to prescribe.

lprzychodzien commented 4 months ago

PR #281 implemented airflow config file which also set sql_alchemy_conn to correctly push all airflow logs into the postgres DB under the airflow database table public.log