mrc-ide / rrq

:runner::runner::runner: Lightweight Redis queues
https://mrc-ide.github.io/rrq/
Other
24 stars 4 forks source link

Prevent accumulation of junk worker logs #90

Closed richfitz closed 1 year ago

richfitz commented 1 year ago

This PR fixes the accumulation of junk worker logs, by interleaving logs from a child process with those from the parent. This should provide a much more consistent view of a task's progress as well as not littering the db with stuff.

There are a few minor bits of change to accommodate this, in particular the movement of where logging happens and a few tests that needed tweaking to account for that. In particular, we were previously deleting the worker log just after writing to it so a few messages were missed.

Previously we created a new worker name as the parent name + random hex. Here we don't do that at all but explicitly track the spawned process is a worker child. I've updated the worker log in a backward compatible way.

Fixes #60

codecov[bot] commented 1 year ago

Codecov Report

Patch coverage: 100.00% and no project coverage change.

Comparison is base (9f3477e) 99.94% compared to head (76e998f) 99.94%.

:exclamation: Current head 76e998f differs from pull request most recent head e8a1038. Consider uploading reports for the commit e8a1038 to get more accurate results

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #90 +/- ## ======================================= Coverage 99.94% 99.94% ======================================= Files 28 28 Lines 1795 1811 +16 ======================================= + Hits 1794 1810 +16 Misses 1 1 ``` | [Impacted Files](https://codecov.io/gh/mrc-ide/rrq/pull/90?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mrc-ide) | Coverage Δ | | |---|---|---| | [R/heartbeat.R](https://codecov.io/gh/mrc-ide/rrq/pull/90?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mrc-ide#diff-Ui9oZWFydGJlYXQuUg==) | `100.00% <100.00%> (ø)` | | | [R/rrq\_controller.R](https://codecov.io/gh/mrc-ide/rrq/pull/90?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mrc-ide#diff-Ui9ycnFfY29udHJvbGxlci5S) | `99.79% <100.00%> (+<0.01%)` | :arrow_up: | | [R/worker.R](https://codecov.io/gh/mrc-ide/rrq/pull/90?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mrc-ide#diff-Ui93b3JrZXIuUg==) | `100.00% <100.00%> (ø)` | | | [R/worker\_run.R](https://codecov.io/gh/mrc-ide/rrq/pull/90?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mrc-ide#diff-Ui93b3JrZXJfcnVuLlI=) | `100.00% <100.00%> (ø)` | | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mrc-ide). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mrc-ide)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.