Build failure with SQLite >=3.11.0

[Arfrever]

[  5%] Generating ../../../kanjidic2.db
cd /var/tmp/portage/app-i18n/tagainijisho-1.0.3/work/tagainijisho-1.0.3_build/src/core/kanjidic2 && ./build_kanji_db -len,fr,ru /var/tmp/portage/app-i18n/tagainijisho-1.0.3/work/tagainijisho-1.0.3 /var/tmp/portage/app-i18n/tagainijisho-1.0.3/work/tagainijisho-1.0.3_build
Could not register katakana tokenizer!
Could not register katakana tokenizer!
Could not register katakana tokenizer!
Could not register katakana tokenizer!
unknown tokenizer: katakana
/var/tmp/portage/app-i18n/tagainijisho-1.0.3/work/tagainijisho-1.0.3/src/core/kanjidic2/BuildKanjiDB.cc: assert condition failed, line 619

From SQLite documentation:

Because of security concerns, SQLite version 3.11.0 only enabled the second form of the fts3_tokenizer() function when the library is compiled with the -DSQLITE_ENABLE_FTS3_TOKENIZER option. In earlier versions it was always available. Beginning with SQLite version 3.12.0, the second form of fts3_tokenizer() can also be activated at run-time by calling sqlite3_db_config(db,SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER,1,0).

Fix:

--- src/sqlite/sqlite3ext.cc
+++ src/sqlite/sqlite3ext.cc
@@ -118,6 +118,10 @@
    sqlite3_stmt *pStmt;
    const char *zSql = "SELECT fts3_tokenizer(?, ?)";

+#ifdef SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER
+   sqlite3_db_config(db, SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER, 1, NULL);
+#endif
+
    rc = sqlite3_prepare_v2(db, zSql, -1, &pStmt, 0);
    if( rc!=SQLITE_OK ){
        return rc;

[Gnurou]

Thanks for reporting this. I guess this means that we need to unconditionally embed SQLite and compile it with -DSQLITE_ENABLE_FTS3_TOKENIZER, lest we give up support for 3.11. That's ok, there are other good reasons to embed it by default anyway, so I'll just do that.