search

counterdata-network / story-processor

Story discovery engine for the Counterdata Network. Grabs relevant stories from various APIs, runs them against bespoke classifier models, post results to a central server.
Apache License 2.0
0 stars 2 forks source link

fire alert if story volume is lower than anticipated #61

Open rahulbot opened 5 months ago

rahulbot commented 5 months ago

At this point we have a good sense of what a "working" system looks like in terms of story volume. After taking a look at the last couple of weeks, I suggest a useful rule would be to say that if the average number of stories retrieved across all platforms over the last four days is below 40,000 then that means we need to look at something.

I propose we add a cron-job that runs that check and fires off a Slack message and/or email if the condition is triggered. This code should be written, so that multiple rules like that could be added at a later point.

I think using a more full-fledged, alerting, package or tool is probably overkill in this case, but I can be convinced otherwise.

math4humanities commented 4 months ago

Hmm... the alert is currently using processed_date, I will investigate why no alert was sent in the past week.

math4humanities commented 2 months ago

Resolved via PR #73