Improve logging and notifications for full pipeline - Githubissues

weecology / portalPredictions

Using various models to forecast abundances at Portal

MIT License

9 stars 8 forks source link

Improve logging and notifications for full pipeline #377

Open ethanwhite opened 3 years ago

ethanwhite commented 3 years ago

The automated pipeline needs to simultaneously accomplish two things:

Be as robust as possible to errors caused by things like upstream data issues
Make sure that errors are quickly identified so that they can be fixed

Here's my first draft of a checklist for things we need to do in this space.

[ ] Implement logging for each step of PortalData
[ ] Implement notifications for failures at any step for PortalData
[ ] Fail PortalData if result of update would break things downstream
[ ] Implement logging in portalcasting
[ ] In portalPredictions implement notifications for failures at any step for portalcasting
[ ] Fail portalPredictions if result of update would result in errors on website

@gmyenni @juniperlsimonis @henrykironde Am I thinking about this right? Are there other things that need to be on the list?