vrischmann / zig-sqlite

zig-sqlite is a small wrapper around sqlite's C API, making it easier to use with Zig.
MIT License
367 stars 49 forks source link

Support large SQL statements #77

Closed lun-4 closed 2 years ago

lun-4 commented 2 years ago

ParsedQuery.query only holds 1KB worth of SQL text, which is not enough for the use case of a migration tool, since after parsing it, we copy the entire query over to that field:

/usr/lib/zig/std/debug.zig:234:14: error: reached unreachable code
    if (!ok) unreachable; // assertion failure
             ^
/usr/lib/zig/std/mem.zig:219:11: note: called from here
    assert(dest.len >= source.len);
          ^
./.zigmod/deps/v/git/github.com/vrischmann/zig-sqlite/branch-master/query.zig:144:17: note: called from here
        mem.copy(u8, &parsed_query.query, &buf);
                ^
./.zigmod/deps/v/git/github.com/vrischmann/zig-sqlite/branch-master/sqlite.zig:1191:44: note: called from here
    return Statement(opts, ParsedQuery.from(query));
                                           ^
./.zigmod/deps/v/git/github.com/vrischmann/zig-sqlite/branch-master/sqlite.zig:528:33: note: called from here
        break :blk StatementType(.{}, query);
                                ^
./.zigmod/deps/v/git/github.com/vrischmann/zig-sqlite/branch-master/sqlite.zig:526:90: note: called from here
    pub fn prepare(self: *Self, comptime query: []const u8) DynamicStatement.PrepareError!blk: {
                                                                                         ^
./src/main.zig:165:41: note: called from here
                    try self.executeOnce(decl_sql);
                                        ^
./src/main.zig:141:46: note: called from here
    pub fn migrateCommand(self: *Self) !void {
                                             ^

SQLite documentation says:

The maximum number of bytes in the text of an SQL statement is limited to SQLITE_MAX_SQL_LENGTH which defaults to 1,000,000,000.

We are some orders of magnitude back on that number... maybe as a paliative we could bump ParsedQuery.query to 32KB? A suggestion that could help it in design-space would be receiving the maximum size of a SQL query as a parameter for the ParsedQuery type itself, and setting it to query.len when we're in Db.prepare, preventing us from always having a 1GB buffer at comptime.