FTS5 support?

[Danlock]

Hello, I'm a fan of your work. I just used it to make a tiny CGO free CLI tool for image search at searmage .

Are there any plans for adding support for the SQLite FTS5 extension?

[ncruces]

It should be enabled (also this), but I've not tested it. Is anything specific not working?

[Danlock]

Oh wow, fantastic. I assumed since mattn/go-sqlite3 requires a build tag to compile it in, it would have been in the ext folder.

It may be good to call that and JSON support in the README.md so it's easier to find for people making a quick comparison to mattn/go-sqlite3.

Thanks for the quick reply!

[ncruces]

OK, I'll review the copy on the main README.

I'm curious about the NUL panic. That likely indicates a place where you passed a “big string” where I was expecting a “small string.”

For most strings, we I use known length APIs that have no limit. Some APIs require NUL termination (e.g. file names?) and I use limits to avoid pointless copies of MB/GBs worth of data.

I may have borked something with too low a limit.

[Danlock]

The specific issue there was this query SELECT path FROM images WHERE path IN array(?) while I was using the array extension.

This was the guilty function.

I probably was using the array extension improperly. But on the other hand that package only has an example for using it with the database/sql driver's, instead of the direct SQLite API.

[ncruces]

Right. Documentation and examples, much to do there.

You need to use stmt.BindPointer, with a Go slice/array.

Still curious on the panic. If you still have the trace, please open an issue. Thanks!

[ncruces]

I assumed since mattn/go-sqlite3 requires a build tag to compile it in, it would have been in the ext folder.

To document the rational behind this: in 2023, WASM doesn't really support dynamic linking, so “native” (implemented in C, or maybe C++/Rust/etc) extensions all need to be statically linked, and bundled into a single WASM blob. Thus, there is little reason to push them to a separate package (unless the WASM becomes unwieldy large).

I've (mostly) bundled everything that I considered useful from SQLite excluding things I can easily implement in Go (like Unicode), also to avoid mixing licenses (SQLite is public domain).

Larger native extensions under different terms would need a separate package, but might become mutually incompatible (matrix of features) so I'd rather avoid that if at all possible.

Things that I do consider I can easily (or should/must) rewrite in Go, are being done bit-by-bit as extensions (there's a ToDo there).

[Danlock]

Thanks for the clarification. All I really needed was unicode, FTS5 and array so those bundling decisions make sense to me. Also it seems like SQLite bundles in FTS5 by default now, so that makes even more sense.