datasette / datasette-tail

Tools for tailing your database
Apache License 2.0
8 stars 0 forks source link

sqlite3 LISTEN NOTIFY and update_hook #2

Open westurner opened 2 weeks ago

westurner commented 2 weeks ago

There are ways to get sqlite to push events for perhaps easier and likely more efficient tailing.

From https://news.ycombinator.com/item?id=41285614 re: similar with postgres, sqlalchemy, celery kombu with just sqlite instead of redis or postgres (which may be useful for datasette-enrichments queue durability) ::

The sqlite3_update_hook() interface registers a callback function with the database connection identified by the first argument to be invoked whenever a row is updated, inserted or deleted in a rowid table. Any callback set by a previous call to this function for the same database connection is overridden.

The second argument is a pointer to the function to invoke when a row is updated, inserted or deleted in a rowid table. The first argument to the callback is a copy of the third argument to sqlite3_update_hook(). The second callback argument is one of SQLITE_INSERT, SQLITE_DELETE, or SQLITE_UPDATE, depending on the operation that caused the callback to be invoked. The third and fourth arguments to the callback contain pointers to the database and table name containing the affected row. The final callback parameter is the rowid of the row. In the case of an update, this is the rowid after the update takes place.

The update hook is not invoked when internal system tables are modified (i.e. sqlite_sequence). The update hook is not invoked when [...]

See also the sqlite3_commit_hook(), sqlite3_rollback_hook(), and sqlite3_preupdate_hook() interfaces.

westurner commented 2 weeks ago

https://news.ycombinator.com/item?id=40842365 :

There are many extensions of SQLite; rqlite, cr-sqlite, postlite, electricsql, sqledge, and also WASM: sqlite-wasm, sqlite-wasm-http

So there must be ways to tail sqlite, for replication and worse synchronization