search

readysettech / readyset

Readyset is a MySQL and Postgres wire-compatible caching layer that sits in front of existing databases to speed up queries and horizontally scale read throughput. Under the hood, ReadySet caches the results of cached select statements and incrementally updates these results over time as the underlying data changes.
https://readyset.io
Other
4.32k stars 120 forks source link

Add a `Packet::Error` packet type and materialize errors #789

Open ethan-readyset opened 12 months ago

ethan-readyset commented 12 months ago

Description

Currently, we have no way to short-circuit a slow upquery or write, which has the potential to destabilize the system, since the domain involved is completely tied up until that upquery/write is finished due to the single-threadedness of domains. In the long term, we'll probably want to investigate ways to make domains multi-threaded, but that would be a major architectural change to ReadySet and has the potential to impact correctness and our consistency model.

In the short term, domain operations have to be fast — a slow operation has the potential to completely stop writes and other upqueries in their tracks. To prevent such slow operations, we should a new Packet::Error type that can be propagated down the graph to a reader node. This would allow us to time out slow domain operations and propagate errors to the reader node, which would store the errors for the keys in the corresponding slots in the reader map. This has the implication that the adapter would have to proxy any queries it gets for a key with an error in the reader map, but that is still much better than our current situation.

Change in user-visible behavior

Yes

Requires documentation change

Yes

gvsg-rs commented 11 months ago

Wait to get some more data from other self-hosting experiments.