search

WiseLibs / better-sqlite3

The fastest and simplest library for SQLite3 in Node.js.
MIT License
5.39k stars 393 forks source link

Question: how to register a FTS3/FTS4 custom tokenizer? #1227

Closed Luminqi closed 2 days ago

Luminqi commented 1 month ago

From the document of SQLite: https://www.sqlite.org/fts3.html, the way to register a custom tokenizer is executing a sql:

SELECT fts3_tokenizer(<tokenizer-name>, <sqlite3_tokenizer_module ptr>)

I am stuck on how to provide the second parameter, which is a buffer storing the pointer to the tokenizer. How can this be done in Node.js?

Luminqi commented 1 month ago

Find a solution:

  1. write tokenizer as a sqlite extension and call SELECT fts3_tokenizer(<tokenizer-name>, <sqlite3_tokenizer_module ptr>) in extension init function
  2. load the extension using loadExtension api

I am not sure if this is the best solution.