xaya / libxayagame

MIT License
21 stars 19 forks source link

Unblocking state reads from SQLite #95

Closed domob1812 closed 4 years ago

domob1812 commented 4 years ago

Read operations of the game state (i.e. GetCustomStateData for GSP RPC methods) may take a long time in some cases, e.g. getbootstrapdata of Taurion. Currently, all operations (block processing and reads) are fully synchronised in libxayagame-based GSPs. This means that e.g. if a GSP is shared (perhaps as Charon server), then one such expensive call will completely block all other calls to it, which is not ideal.

Thus we should make read operations "unsynchronised", especially for SQLite-based games (which have large states). These are the steps we need for that:

1) Non-SQLite-based games should release the lock on Game as soon as the extraction-callback of GetCustomStateData is invoked (since it then just processes the already-extracted state data). 2) SQLite-based games should open a new read-only database connection with a snapshot, and then release the lock before the actual state extraction from the snapshot is done in the callback. 3) Step 2 only works if the current state of the database is flushed, i.e. there is no open batched transaction (as e.g. during catch-up). So we need to make sure to verify that the snapshot's state matches the block hash we got from Game, and revert back to the original database connection if not (and keep the lock then).

domob1812 commented 4 years ago

Note that we also have to refactor the way prepared statements are managed if we want to use potentially multiple SQLite connections. In other words, caching of prepared statements should be done with a helper object that holds an explicit sqlite3* (perhaps also encapsulating a DB snapshot), and not directly through SQLiteGame.