maxClauseCount exception in Content Selector

[alansemenov]

Reported in support ticket

Original message: If you have a ContentSelector browsing a content with a lot of direct child content (thousands) an error happens. If you have treeMode on and navigate down to and expand the content with a lot of direct childs, it happens. Also happens when not using treeMode after scrolling for a while.

Content Selector with a very basic allowPath config (see below) gives an exception on a big amount of content, even though paging only fetches 15 items at a time. The reason seems to be that we do aggregation by path in ES query (see query on the top of the stack trace) which hits the limitation. This is not something that should be configurable by a user, we have to remove this limitation or allow max reasonable number.

<input name="id" type="ContentSelector">
<label i18n="content-choose">Velg innhold</label>
<occurrences minimum="1" maximum="1"/>
<config>
<allowPath>${site}/*</allowPath>
</config>
</input>

_12:11:52.955 ERROR c.e.x.j.i.e.JsonExceptionMapper - Search request failed after [30s], query: [{ "from" : 0, "size" : 15, "query" : { "bool" : { "must" : { "bool" : { "should" : [ { "term" : { "_path" : { "value" : "/content/no/forskning/forskningspublikasjoner/rok-regional-policy-and-northeast-asian-security", "_name" : "path" } } },

Caused by: org.apache.lucene.search.BooleanQuery$TooManyClauses: maxClauseCount is set to 1024 at org.apache.lucene.search.BooleanQuery$Builder.add(BooleanQuery.java:137) at org.apache.lucene.search.BooleanQuery$Builder.add(BooleanQuery.java:125) at org.elasticsearch.index.query.BoolQueryParser.parse(BoolQueryParser.java:171) at org.elasticsearch.index.query.QueryParseContext.parseInnerQuery(QueryParseContext.java:250) at org.elasticsearch.index.query.BoolQueryParser.parse(BoolQueryParser.java:76) at org.elasticsearch.index.query.QueryParseContext.parseInnerQuery(QueryParseContext.java:250) at org.elasticsearch.index.query.IndexQueryParserService.innerParse(IndexQueryParserService.java:324) at org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:224) at org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:219) at org.elasticsearch.search.query.QueryParseElement.parse(QueryParseElement.java:33) at org.elasticsearch.search.SearchService.parseSource(SearchService.java:856) at org.elasticsearch.search.SearchService.createContext(SearchService.java:667) at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:633) at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:472) at org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryFetchTransportHandler.messageReceived(SearchServiceTransportAction.java:392) at org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryFetchTransportHandler.messageReceived(SearchServiceTransportAction.java:389) at org.elasticsearch.transport.TransportRequestHandler.messageReceived(TransportRequestHandler.java:33) at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:77) at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:378) at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted

[rymsha]

Most important part of the error comes after comma: {term: path: value : /a}, {term: path: value : /b}, {term: path: value : /c}, ... and it continues more than 1024 times.

This is analog of antipattern SQL: SELECT * FROM t WHERE path IN ['/a', '/b', '/c', ... ]

[sigdestad]

How do we change this? Make it a regular startswith Query instead?

[rymsha]

Workaround is to set indices.query.bool.max_clause_count to a greater value in com.enonic.xp.elasticsearch.cfg.

[rymsha]

related #5688