search

mikemccand / stargazers-migration-test

Testing Lucene's Jira -> GitHub issues migration
0 stars 0 forks source link

Add a multiplexing TokenFilter [LUCENE-8327] #327

Open mikemccand opened 6 years ago

mikemccand commented 6 years ago

Following on from LUCENE-8273, and as a prerequisite of LUCENE-8308, it would be useful to have a TokenFilter that takes a number of child filters, and repeats its incoming stream, applying each filter in turn. So for example, you could keep the original term, output ngrams, and apply stemming, all in the same token stream.

Legacy Jira details

LUCENE-8327 by Alan Woodward (@romseygeek) on May 22 2018 Attachments: LUCENE-8327.patch

mikemccand commented 6 years ago

Here's an initial patch sketching out the idea. It still needs a filter factory and adding into CustomAnalyzer, and it would be nice to somehow add it into TestRandomChains. A caveat is that I don't think this will work with tokenfilters that need to read ahead, like SynonymFilter.

[Legacy Jira: Alan Woodward (@romseygeek) on May 22 2018]