This PR adds a new Pool type which is helpful for relieving lock contention in a many-reads few-writes scenario. It is akin to a database cluster, with one "leader" interpreter used for writes and many "follower" interpreters used for reads. Reads are currently distributed to the follower interpreters in a round-robin fashion. After a write, the leader's wasm memory is copied to the followers'. This allows for a simple implementation but could be improved with more specialized memory copying.
Additionally, the locking strategy for QueryOnce was improved. The lock will be held from query creation until it is finished (and its memory is freed), which prevents other queries "barging in" and potentially exhausting the wasm memory space before the query has a chance to execute. In effect this means you can throw an unlimited amount of concurrent QueryOnce calls at an interpreter and it should be fine.
Benchmark of 100 concurrent queries against a Pool vs single interpreter (on a 12 CPU computer).
This PR adds a new
Pooltype which is helpful for relieving lock contention in a many-reads few-writes scenario. It is akin to a database cluster, with one "leader" interpreter used for writes and many "follower" interpreters used for reads. Reads are currently distributed to the follower interpreters in a round-robin fashion. After a write, the leader's wasm memory is copied to the followers'. This allows for a simple implementation but could be improved with more specialized memory copying.Additionally, the locking strategy for QueryOnce was improved. The lock will be held from query creation until it is finished (and its memory is freed), which prevents other queries "barging in" and potentially exhausting the wasm memory space before the query has a chance to execute. In effect this means you can throw an unlimited amount of concurrent QueryOnce calls at an interpreter and it should be fine.
Benchmark of 100 concurrent queries against a Pool vs single interpreter (on a 12 CPU computer).
We get around a ~4.3x speedup, not bad.