This is a quick fix while we work out a better strategy long term.
Currently each tx is written to each active shape in lock-step: new txns are only requested from the replication stream once all shapes have written the last one to disk.
We currently have a problem with timeouts because the write function is done via a GenServer.call/3 with a default timeout of 5s. So if there are a 1000 shapes, each taking ~5ms to write, then this call times out.
This is trivial to fix, instead of having an overall timeout for writing to all shapes, we move the timeout handling to the storage, so each individual shape will crash if its write takes too long, and the overall write call just sets a timeout of :infinity and waits.
This is a quick fix while we work out a better strategy long term.
Currently each tx is written to each active shape in lock-step: new txns are only requested from the replication stream once all shapes have written the last one to disk.
We currently have a problem with timeouts because the write function is done via a
GenServer.call/3with a default timeout of 5s. So if there are a 1000 shapes, each taking ~5ms to write, then this call times out.This is trivial to fix, instead of having an overall timeout for writing to all shapes, we move the timeout handling to the storage, so each individual shape will crash if its write takes too long, and the overall write call just sets a timeout of
:infinityand waits.