When the agent restarts, we might end up in a situation where we are running a check too often. For example, if a check is configured to run once every 10 minutes, and the check ran 1 minute ago, if we run immediately, we will have two executions within that 10 minute window.
Since we have to publish samples once every two minutes, we cannot wait for 9 minutes, because we end up with a gap. Instead, do run immediately to avoid that.
But if the check ran 9 minutes ago, we can align with the expectation by waiting for 1 minute instead of a random value. Presumably the check ran 9 minutes ago, and the sample was replicated 7, 5, 3 and 1 minutes ago. If we wait for 1 minute, we would end up running the check when it was expected to run.
In order to actually fix this issue the agents would have to persist data across runs. It might be possible to do this by offloading publishing to another service.
When the agent restarts, we might end up in a situation where we are running a check too often. For example, if a check is configured to run once every 10 minutes, and the check ran 1 minute ago, if we run immediately, we will have two executions within that 10 minute window.
Since we have to publish samples once every two minutes, we cannot wait for 9 minutes, because we end up with a gap. Instead, do run immediately to avoid that.
But if the check ran 9 minutes ago, we can align with the expectation by waiting for 1 minute instead of a random value. Presumably the check ran 9 minutes ago, and the sample was replicated 7, 5, 3 and 1 minutes ago. If we wait for 1 minute, we would end up running the check when it was expected to run.
In order to actually fix this issue the agents would have to persist data across runs. It might be possible to do this by offloading publishing to another service.
Fixes: #739