NIAID-Data-Ecosystem / nde-crawlers

Harvesting infrastructure to collect and standardize dataset and computational tool metadata
Apache License 2.0
0 stars 0 forks source link

Create a reporter to track the status of a data build #26

Open flaneuse opened 2 years ago

flaneuse commented 2 years ago

Create a wrapper which makes it easy to check on the health of the Hub updates upon scheduled updates, rather than having to dig through the logs to know if updates failed/succeeded. Should focus on alerting us when something goes wrong and requires intervention. Julia built a slackbot off the /metadata endpoint which has been useful for tracking the size of each outbreak.info source and the last day it was updated:

Screen Shot 2022-04-14 at 5 34 04 PM

It'd be cool if something like this was also tied to error states that would require work during an autorelease.