TooManyClauses Exception with wildcard queries

[Shazwazza]

Based on the update for https://github.com/Shazwazza/Examine/pull/89 we now have a new problem with larger indexes that contain a lot of prefixed terms.

The change to #89 has been reverted - see below for work arounds

[Shazwazza]

I've added a work around for now, it's not ideal but there isn't an ideal solution to this problem in Lucene - unless we can use the very latest version (which we cannot).

In larger indexes when there are many prefixed terms this exception will occur a lot based on the changes made in #89.

As it turns out, we probably shouldn't have set this code to execute in the core since this functionality can just be set using the Examine APIs. I might revert this addition and have people rely on using the Examine APIs to set this instead if required since this might cause more problems than it's worth.

You can do this with the Examine APIs before the v0.1.87 changes:

var criteria = (LuceneSearchCriteria)searcher.CreateSearchCriteria(); 
criteria.QueryParser.SetMultiTermRewriteMethod(MultiTermQuery.SCORING_BOOLEAN_QUERY_REWRITE);

and the continue building up your query

[Shazwazza]

I've publisized the work around class so if you wish to automatically ignore these exceptions when trying th use SCORING_BOOLEAN_QUERY_REWRITE (consider performance problems here though because if you are getting these exceptions it already means peformance is a problem) you can use this code:

var criteria = searcher.CreateSearchCriteria(); 
var luceneCriteria = (LuceneSearchCriteria)criteria;
luceneCriteria.QueryParser.SetMultiTermRewriteMethod(BaseLuceneSearcher.ErrorCheckingScoringBooleanQueryRewriteInstance);
//Continue using the 'criteria' object to build up the query

[ismailmayat]

@Shazwazza cool thanks for this, apologie for blowing up our lol. Btw you coming UK fest? Wanted to talk about looking at updating examine lucene to 4.8 see if cogworks can help? Appreciate it will be big job.

Regards

Ismail

[Shazwazza]

Unfortunately i can't make it.

Changing Examine to 4.8 will be a dream. Most of the v2 work will be updated, but here's what I'd like to do to get there:

• v2 branch has a back ported version of SearchManager, NRTManager, etc... and all of that plumbing is to be removed since 4.8 has this built correctly
• v2 branch has all sorts of faceting logic, this is to be moved to a separate project and Examine updated to support adding this functionality via APIs some how - but we can decide that it's just not important if that takes to long and leave it out
• Much of the v2 stuff will remain