harvard-lil / perma

Indelible links
411 stars 70 forks source link

Alert if something goes wrong with the Internet Archive task count #3413

Closed rebeccacremona closed 9 months ago

rebeccacremona commented 9 months ago

ENG-417

Something got scrambled, back in July, with Perma's internal record of how many Internet Archive tasks we currently have in flight: we think things got out-of-whack during a phased reboot of the workers.

Our records ended up in a state where, we thought we had more tasks in flight than is permitted, and so, we refrained from making any more uploads since.

We don't think this will happen again, because of a change in strategy: we will now go into maintenance mode before making changes like this.

But, this PR adds another level of safety: if we get into the same situation again, the upload task will now log to the error log.

codecov[bot] commented 9 months ago

Codecov Report

Attention: 3 lines in your changes are missing coverage. Please review.

Comparison is base (79ad0dd) 68.65% compared to head (fd29e45) 68.62%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## develop #3413 +/- ## =========================================== - Coverage 68.65% 68.62% -0.03% =========================================== Files 53 53 Lines 7233 7236 +3 =========================================== Hits 4966 4966 - Misses 2267 2270 +3 ``` | [Files](https://app.codecov.io/gh/harvard-lil/perma/pull/3413?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=harvard-lil) | Coverage Δ | | |---|---|---| | [perma\_web/perma/celery\_tasks.py](https://app.codecov.io/gh/harvard-lil/perma/pull/3413?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=harvard-lil#diff-cGVybWFfd2ViL3Blcm1hL2NlbGVyeV90YXNrcy5weQ==) | `47.71% <0.00%> (-0.11%)` | :arrow_down: |

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.