pennsignals / dsdk

Data Science Deploy Kit
MIT License
8 stars 7 forks source link

Add pre and post run steps #55

Closed jlubken closed 3 years ago

jlubken commented 3 years ago

Corey was looking for a way to add a verify/validation step after the pipeline runs to examine postgres predictions.

This cannot be a task in the pipeline because the predictions, and run are only written toipostgres after the pipeline is completed.

The intent is to provide an optional post-pipeline task/tasks that can verify/validate without the race conditions and complexity invited by:

@mdbecker Let me know if you have other ideas to handle validation systems test: ingest -> predict -> postgres. I don't want to test the whole integration: ingest -> predict -> postgres -> epic. The validation/verification is to check the model (not the entire integration pipeline). Ideally we could exclude ingest errors as well, but not at this point.

jlubken commented 3 years ago

Completed by overriding Service.call and adding new console script endpoints to the Service class.