Examine GetFieldInternalQuery error

jplatfordquba commented 1 year ago

A site of ours (Umbraco v10.2.0) experienced this error yesterday and we haven't been able to reproduce it on our development or test environments. A site reboot resolved the issue and it is yet to return. It appeared on all pages that use Examine for search. The error occurred regardless of search term/filter.

System.NullReferenceException: Object reference not set to an instance of an object. at Lucene.Net.QueryParsers.Classic.QueryParser.Jj_add_error_token(Int32 kind, Int32 pos) at Lucene.Net.QueryParsers.Classic.QueryParser.Jj_scan_token(Int32 kind) at Lucene.Net.QueryParsers.Classic.QueryParser.Jj_3R_2() at Lucene.Net.QueryParsers.Classic.QueryParser.Jj_3_1() at Lucene.Net.QueryParsers.Classic.QueryParser.Jj_2_1(Int32 xla) at Lucene.Net.QueryParsers.Classic.QueryParser.Clause(String field) at Lucene.Net.QueryParsers.Classic.QueryParser.Query(String field) at Lucene.Net.QueryParsers.Classic.QueryParser.TopLevelQuery(String field) at Lucene.Net.QueryParsers.Classic.QueryParserBase.Parse(String query) at Examine.Lucene.Search.LuceneSearchQueryBase.GetFieldInternalQuery(String fieldName, IExamineValue fieldValue, Boolean useQueryParser) at Examine.Lucene.Search.LuceneSearchQuery.Search(QueryOptions options)

nikcio commented 1 year ago

Hi @jplatfordquba if there's going to be any chance of figuring out the issue we need some more information. Examples:

What precise Examine version in use.
Examples of queries that caused the issue and expected results of these queries (Not the data but maybe amount of elements returned)
Do you have any custom code to Examine/Lucene related code. I.E custom index, custom searcher or similar.

jplatfordquba commented 1 year ago

Hi @nikcio

Examine version 3.0.0
All attempts to use pages with Examine used for searching errored, regardless of query. Essentially the only thing it was filtering for were document types (News doc types for the News search for example).
No custom indexes, just the ExternalIndex default to Umbraco

Snippet below:


var criteria = searcher.CreateQuery(IndexTypes.Content, BooleanOperation.And);
IBooleanOperation examineQuery;
examineQuery = criteria.GroupedOr(new string[] { "__NodeTypeAlias" }, searchableNodeTypes);
var results = examineQuery.Execute();

nzdev commented 1 year ago

@jplatfordquba the value / type of searchableNodeTypes needs to be included to help diagnose. Also the query string that can be obtained from the debugger by inspecting the value of examineQuery.

danlb2000 commented 12 months ago

We are seeing the same problem as @jplatfordquba. It is the exact same symptoms with the exact same stack trace in Umbraco 10.6.1. All queries will work fine for days, but all of a sudden queries that worked fine before now fail. Every query we do will fail until we restart the app pool and then everything will work fine again. We run two load balanced server and the failure will randomly happen on one or the other.

Shazwazza commented 12 months ago

As this error is coming from Lucene code, I might suggest reporting this issue on the Lucene repo https://github.com/apache/lucenenet

But like @nzdev mentions, without all of the information its pretty hard to diagnose the issue without steps to replicate.

danlb2000 commented 12 months ago

Unfortunately we are unable to determine what steps are needed to replicate it. It happens pretty randomly.

Shazwazza commented 12 months ago

Is it generally from the same query? If so, can you share the entire query and how you are creating it? I wonder if we can setup a test to continuously execute it and see if it eventually fails.

danlb2000 commented 12 months ago

Is it generally from the same query? If so, can you share the entire query and how you are creating it? I wonder if we can setup a test to continuously execute it and see if it eventually fails.

We query the index in a couple different areas of our site and once it starts failing it fails in all of them. This is probably the most straight forward way we query it. This is for a general index page so no user input is involved, all the parameters are statically determined. Note this code is slightly simplified.

if (!_examineManager.TryGetIndex(UmbracoIndexes.ExternalIndexName, out var index) || !(index is IUmbracoIndex umbIndex)) { throw new InvalidOperationException($"No index found by name ExternalIndex or is not of type {typeof(IUmbracoIndex)}"); } QueryOptions queryOptions = new QueryOptions(0, 3000); string[] docTypes = docTypeList.ToArray(); // Gets populated with a list of doc types var resultsAsSearchItems = index .Searcher .CreateQuery(IndexTypes.Content) .Field("__Published_en", "y") .And() .GroupedOr(new[] { "__NodeTypeAlias" }, docTypes) .Execute(queryOptions);

This is the stack trace it produces when the problem occurs. We have a more complex query that produces the exact stack trace that @jplatfordquba reported

System.NullReferenceException: Object reference not set to an instance of an object. at Lucene.Net.QueryParsers.Classic.ParseException.Initialize(Token currentToken, Int32[][] expectedTokenSequences, String[] tokenImage) at Lucene.Net.QueryParsers.Classic.QueryParser.GenerateParseException() at Lucene.Net.QueryParsers.Classic.QueryParser.Jj_consume_token(Int32 kind) at Lucene.Net.QueryParsers.Classic.QueryParser.Clause(String field) at Lucene.Net.QueryParsers.Classic.QueryParser.Query(String field) at Lucene.Net.QueryParsers.Classic.QueryParser.TopLevelQuery(String field) at Lucene.Net.QueryParsers.Classic.QueryParserBase.Parse(String query) at Examine.Lucene.Search.LuceneSearchQueryBase.GetFieldInternalQuery(String fieldName, IExamineValue fieldValue, Boolean useQueryParser) at Examine.Lucene.Search.LuceneSearchQuery.Search(QueryOptions options)

danlb2000 commented 11 months ago

Another interesting piece of information on this. We removed IndexTypes.Content from CreateQuery(IndexTypes.Content) since in our scenario it really wasn't needed and when we did this searches no longer fail. I was able to confirm this since we missed that change in one place and when the problem re-occurred it only failed on the function where we didn't remove it, it worked on in other areas.