search

rusqlite / rusqlite

Ergonomic bindings to SQLite for Rust
MIT License
3.05k stars 351 forks source link

Custom FTS5 tokenizer support #689

Open thomcc opened 4 years ago

thomcc commented 4 years ago

Have poked at adding this a few times, filing an issue for future reference, I don't think its a high priority feature. Eventually I'll get back to it.

SQLite documentation: https://www.sqlite.org/fts5.html#custom_tokenizers

Swift's GRDB has it and I've always been jealous of it: https://github.com/groue/GRDB.swift/blob/master/Documentation/FTS5Tokenizers.md

ColonelThirtyTwo commented 3 years ago

Wrote a module to do this safely. Feel free to steal: https://gist.github.com/ColonelThirtyTwo/3dd1fe04e4cff0502fa70d12f3a6e72e