Previously shotover's design did not allow for a true implementation of flush_when_millis_since_last_flush.
There was no way for the transform to trigger a flush all on its own, it had to wait for a new request to come in before the flush could be triggered.
This results in easily triggered poor behavior, for example:
flush_when_millis_since_last_flush set to 1000 (1 second)
client sends 1 request
1 hour passes and no more requests are sent
flush never occurs and request is just stuck in shotover.
However shotover now supports this kind of functionality via force_run_chain.
So we should reimplement the flush_when_millis_since_last_flush logic of Coalesce to setup a tokio task in the background that will trigger a flush after the time has passed.
This should be done by:
setting up the task in CoalesceConfig::get_builder
the task runs a tokio::sleep(Duration::from_millis(flush_when_millis_since_last_flush)).await in a loop.
whenever the sleep elapses: trigger force_run_chain and send a message to the transform instance to let it know it needs to flush.
every time a flush occurs send the task a message to tell it to restart its timer
Previously shotover's design did not allow for a true implementation of flush_when_millis_since_last_flush. There was no way for the transform to trigger a flush all on its own, it had to wait for a new request to come in before the flush could be triggered.
This results in easily triggered poor behavior, for example:
However shotover now supports this kind of functionality via
force_run_chain. So we should reimplement theflush_when_millis_since_last_flushlogic of Coalesce to setup a tokio task in the background that will trigger a flush after the time has passed.This should be done by:
CoalesceConfig::get_buildertokio::sleep(Duration::from_millis(flush_when_millis_since_last_flush)).awaitin a loop.force_run_chainand send a message to the transform instance to let it know it needs to flush.