Update project (#27, 32)

tl-its-umich-edu / unizin-validation

Unizin Validation Scripts

1 stars 4 forks source link

Update project (#27, 32) #33

Closed ssciolla closed 1 year ago

ssciolla commented 1 year ago

Resolves #27, #32

To Do:

[x] Bump to Python 3.10
[x] Update pip packages
[x] Adopt docker-compose
[x] Use sqlalchemy.engine.Engine instead of param objects to suppress warning
[x] Refactor DBConnManager to be re-usable, use context manager
[x] Adopt datetime.timezone, match/case
[x] Configure mypi and add TypedDicts for the query, job, and data source dictionaries

ssciolla commented 1 year ago

Just a minor suggestion: inside the jobs.py, can we put the UDP udp_context_store_view_counts query towards the last of the list, so that all other three UDW queries will be run first before this UDP context_store query?

"Unizin": { "full_name": "Unizin Daily Status Report", "queries": [ "unizin_metadata", "udw_table_counts", "udp_context_store_view_counts", "number_of_courses_by_term" ] }

Sure, I can do that

ssciolla commented 1 year ago

Just a minor suggestion: inside the jobs.py, can we put the UDP udp_context_store_view_counts query towards the last of the list, so that all other three UDW queries will be run first before this UDP context_store query? "Unizin": { "full_name": "Unizin Daily Status Report", "queries": [ "unizin_metadata", "udw_table_counts", "udp_context_store_view_counts", "number_of_courses_by_term" ] }

Sure, I can do that

Done, @zqian. See https://github.com/tl-its-umich-edu/unizin-validation/pull/33/commits/d54fb3cb81f7cce3696c18ec2eb6c9ecbba4be4f

ssciolla commented 1 year ago

@jonespm, do you want to review this or should I merge?

ssciolla commented 1 year ago

Thanks for the reviews! Merging now...