IreneKnapp / direct-sqlite

MIT License
35 stars 54 forks source link

Allow opening sqllite from bytestring via sqllite3_deserialize directive #97

Closed meghfossa closed 8 months ago

meghfossa commented 2 years ago

As a library consumer, I want to open sqllite3 from byte string and perform operations (as supported by library).

I think sqllite3_deserialize directive can enable this.

Usecase

When working within (non-standard filesystem) - S3, tarball, zip, etc - We usually retrieve byte string of SQLite file. We are interested in working with this.

Workaround

Right now, the workaround is: to persist this SQLite file in some temp file, do necessary operations and remove temp file.

Reference

https://www.sqlite.org/c3ref/deserialize.html

jchia commented 2 years ago

sqlite3_deserialize() looks complicated. Any idea how to implement it properly in direct-sqlite? What are some use cases? From the documentation, I'm also not sure who owns pData after the call to sqlite3_deserialize().

Design proposal/discussion or PR welcome.

meghfossa commented 2 years ago

I can add some use cases here:

When working within (non-standard filesystem) - S3, tarball, zip, etc - We usually retrieve byte string of SQLite file. We are interested in working with this. Right now, the workaround is: to persist this SQLite file in some temp file, do necessary operations and remove temp file.

jchia commented 2 years ago

For this use case, I don't understand why sqlite3_deserialize() even requires an existing database connection to disconnect first. Whoever implements sqlite3_deserialize() in direct-sqlite would have to understand what it really does. The documentation page is rather unclear. E.g. it talks about "serialization buffer" exactly once without explaining what it is or perhaps which variable it refers to.

jchia commented 8 months ago

PR welcome, possibly preceded by design/interface discussion.