ikawaha / kagome

Self-contained Japanese Morphological Analyzer written in pure Go
MIT License
818 stars 53 forks source link

Add working examples of full-text-search of SQLite3 #309

Closed KEINOS closed 1 year ago

KEINOS commented 1 year ago

This PR is a part of #299 which is an example of using Kagome as a tokenizer for full-text-search, a.k.a. FTS4, of SQLite3.

ikawaha commented 1 year ago

sugoi!

KEINOS commented 1 year ago

F.Y.I.

This PR was inspired while reading the below book by @mattn san.

KEINOS commented 1 year ago

I just noticed that the below two query at line 76 and 77 in main.go, the column name words and content should be in reverse for the usage.

_, err = db.Exec(`
        CREATE TABLE IF NOT EXISTS contents_fts(docid INTEGER PRIMARY KEY AUTOINCREMENT, words TEXT);
        CREATE VIRTUAL TABLE IF NOT EXISTS fts USING fts4(content, tokenize=unicode61 "remove_diacritics=2");
    `)

I will 追いPush the changes.

KEINOS commented 1 year ago

Done!

@ikawaha Plz review it when you got free time.

mattn commented 1 year ago

sugoi!

KEINOS commented 1 year ago

@ikawaha Sorry, I should've PRed to the develop branch. Won't happen again. 🙏

ikawaha commented 1 year ago

@KEINOS Never mind, I just forgot to create a develop branch :p. PRs are always welcome!

ikawaha commented 1 year ago

@KEINOS LGTM 👍

It would be better if the example sentences were easy to understand the results of morphological analysis. Please review the PR for https://github.com/KEINOS/Fork_kagome/pull/3 and merge if you would like.

KEINOS commented 1 year ago

@ikawaha

I merged your PR, thank you!

Indeed, it got way better to understand about the analysis. I added some other search words to clarify more. Please merge if it LGTY!

ikawaha commented 1 year ago

Thank you!