search

resilar / sqleet

SQLite3 encryption that sucks less
The Unlicense
375 stars 55 forks source link

Loadable as a runtime extension? #5

Closed JoshuaWise closed 5 years ago

JoshuaWise commented 7 years ago

Sqlite3 provides the ability to load extensions at runtime with the load_extension() function. Is it possible, or will it ever be possible to load this encryption library at runtime? Runtime loading is useful for binding to dynamic languages like Python or JavaScript

resilar commented 7 years ago

All encryption extensions that I know (official SEE plugin, SQLCipher, ...) use SQLite3's codec support to implement encryption, which requires defining SQLITE_HAS_CODEC=1 at the compile time in addition to providing sqlite3_key(), sqlite3_rekey() functions and a few others. Standard SQLite3 is built without the codec support so there is not much we can do unless the SQLite3 devs decide to expose the codec interface for runtime extensions in future releases (not very likely IMO). You can read SQLCipher's excuses from the bottom of their design page.

However, that being said, I think it is still possible and perfectly reasonable to implement transparent, on-the-fly, zero-configuration encryption for SQLite3 as a runtime extension. The idea is to ditch the codec interface altogether and register a runtime VFS extension instead. In fact, this may even be a superior approach allowing us to encrypt temporary files and use deterministic nonces, which have proven to be near impossible to do via the codec interface. sqleet is already portable across multiple versions of SQLite, but a runtime extension implementation would make it even more so.

To my knowledge, no one has ever written SQLite3 encryption as a runtime VFS extension, so there might be some unexpected hurdles (e.g., reserving space for nonces/MACs may turn out to be difficult). sqleet is happy to be the first, but it will take weeks before I have time to spend on this...

JoshuaWise commented 7 years ago

This would be a huge win for the Sqlite3 community, IMO. I'd be very excited to see a runtime VFS extension. I'm the author of a relatively loved sqlite3 bindings library for Node.js, and I've been wanting to add encryption support for a while, but I've yet to find a solution that doesn't require a modified compilation process (which becomes a pain in Node.js).

resilar commented 6 years ago

I have explored this approach a bit, and it seems like there is no clean solution for reserving space for nonces and MACs. In future, I might release a proof of concept with flashy warnings that the loadable extension does not provide authentication, but I'd not recommend anyone using it for anything serious. There are few things that we could do if we allow extra page reads/writes for small performance penalty, but I have not looked into those yet...

JoshuaWise commented 5 years ago

@resilar Thanks for exploring this. I'll consider the issue resolved (nofix).