I'm trying to figure out how to add a custom tokenizer when indexing on model save. I have records with id's that use dashes, like 15-00023, and I need to be able to index and search with the dash. I know how to add it directly to tntsearch if I create my own import command to index, but that will only help me first run and not when I'm saving the models. any thoughts?
I have a custom tokenizer
` use TeamTNT\TNTSearch\Support\TokenizerInterface;
class DashTokenizer implements TokenizerInterface {
public function tokenize($text) {
return preg_split("/[^\p{L}\p{N}-]+/u", strtolower($text), -1, PREG_SPLIT_NO_EMPTY);
}
}
`
When I'm searching, I'm using the TntSearch and setting the tokenizer.
$tnt = new TNTSearch; $driver = config('database.default'); $config = config('scout.tntsearch') + config("database.connections.$driver"); $tnt->loadConfig($config); $tnt->setDatabaseHandle(app('db')->connection()->getPdo()); $tnt->selectIndex($index); $tnt->setTokenizer(new DashTokenizer); $res = $tnt->searchBoolean($query, 12);
I'm trying to figure out how to add a custom tokenizer when indexing on model save. I have records with id's that use dashes, like 15-00023, and I need to be able to index and search with the dash. I know how to add it directly to tntsearch if I create my own import command to index, but that will only help me first run and not when I'm saving the models. any thoughts?
I have a custom tokenizer ` use TeamTNT\TNTSearch\Support\TokenizerInterface;
class DashTokenizer implements TokenizerInterface {
} `
When I'm searching, I'm using the TntSearch and setting the tokenizer.
$tnt = new TNTSearch; $driver = config('database.default'); $config = config('scout.tntsearch') + config("database.connections.$driver"); $tnt->loadConfig($config); $tnt->setDatabaseHandle(app('db')->connection()->getPdo()); $tnt->selectIndex($index); $tnt->setTokenizer(new DashTokenizer); $res = $tnt->searchBoolean($query, 12);