simonw
commented
1 year ago This is really interesting. I'd love to see a working prototype of this.
I'm a bit cautious about introducing required dependencies on things in Rust, just because it can make it so much harder to install things (and makes compatibility with Pyodide for things like Datasette Lite harder), but I'm definitely on board with it as an optional dependency for people who want to use it.
I noticed the chat transcript suggests using delta encodings to save space. For keeping track of tables with large values that change this could be (much?) more space efficient than the current mask-based approach. This can be implemented using Fossil's delta encoding algorithm, which is said to be very good. It is available for SQLite via a Rust library fossil-delta (also with Python bindings).