Long range filtering doesnt work

[bielu]

Describe the bug Generated query is not working through examinex, but work in azure search query panel

License ID Your ExamineX license Id (found in your license file) if you are running a licensed version. 5f2da131-2ba2-49ca-a7ba-a4d0eb087469

To Reproduce Steps to reproduce the behavior: 1) setup any field which is indexed as long and try to use range query

Expected behavior A clear and concise description of what you expected to happen.

Screenshots If applicable, add screenshots to help explain your problem.

Versions

• ExamineX version 6.0.5
• Umbraco version 13.2.2
• Examine version 3.2

Additional context That is interesting one as it didnt happen on 6.0.1, so we have a new one!

[AndrewBarta-SC]

@bielu thanks for sending this through. We'll try to get this replicated and get a fix out by Tuesday next week. I'll be sure to reach out if we have any clarifications needed while we work through this.

[bielu]

@AndrewBarta-SC any updates on this?

[Shazwazza]

Hi @bielu - Our tests for managed long range queries are working. From the looks of it, you have quite a specific query across many fields being executed. Can you help provide some code to replicate the issue?

Running a simple RangeQuery does it work? Though, I don't think this would be isolated to just long, it would probably be the same for any RangeQuery since those all get re-formatted to OData filters.

Also, out of curiosity, did 6.0.1 work as expected with the exact same query you are executing for this today?

Would you also be able to provide the text output of your log to help further analyze?

[bielu]

Yes 6.0.1 works no problem 6.0.5 cause this issue. About the code it might be hard as we using abstraction on top which builds examine query on fly based on query.

[bielu]

About logs I will deliver you text logs tomorrow.

[Shazwazza]

Ok thanks, do you know if 6.0.1 was the last one that worked or did 6.0.2/3/4 work?

[bielu]

I do not know that, if I will have time tomorrow I will test it with other versions.

[bielu]

@Shazwazza I forgot about it, for now, example log from umbraco log viewer, if you want I can try get you original log file instead, but need check it with client first. Expection detail:

Azure.RequestFailedException: Failed to parse query string at line 1, column 8. See https://aka.ms/azure-search-full-query for supported syntax.
Status: 400 (Bad Request)
ErrorCode: InvalidRequestParameter

Content:
{"error":{"code":"InvalidRequestParameter","message":"Failed to parse query string at line 1, column 8. See https://aka.ms/azure-search-full-query for supported syntax.","details":[{"code":"InvalidSearchSyntax","message":"Failed to parse query string at line 1, column 8. See https://aka.ms/azure-search-full-query for supported syntax."}]}}

Headers:
Cache-Control: no-cache,no-store
Pragma: no-cache
Server: Microsoft-IIS/10.0
client-request-id: 9377793e-e8e5-468c-bbc4-8e7a1f7ea9ed
x-ms-client-request-id: 9377793e-e8e5-468c-bbc4-8e7a1f7ea9ed
request-id: 9377793e-e8e5-468c-bbc4-8e7a1f7ea9ed
elapsed-time: 34
Strict-Transport-Security: REDACTED
Date: Thu, 27 Jun 2024 08:18:54 GMT
Content-Length: 341
Content-Type: application/json; charset=utf-8
Content-Language: REDACTED
Expires: -1

   at Azure.Search.Documents.SearchClient.SearchInternal[T](SearchOptions options, String operationName, Boolean async, CancellationToken cancellationToken)
   at Azure.Search.Documents.SearchClient.SearchInternal[T](String searchText, SearchOptions options, Boolean async, CancellationToken cancellationToken)
   at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted[T](Task`1 task)
   at Azure.Search.Documents.SearchClient.Search[T](String searchText, SearchOptions options, CancellationToken cancellationToken)
   at ExamineX.AzureSearch.AzureSearchResults.b.A()
   at ExamineX.AzureSearch.AzureSearchResults.A[A](String, Func`1, Func`1)

Message

"Search failed for query ((search.ismatchscoring('x__IndexType:content', '', 'full', 'any')) and search.ismatchscoring('x__Published:y', '', 'full', 'any') and not (search.ismatchscoring('x__Path:-20', '', 'full', 'any')) and search.ismatchscoring('x__Published_en_gb:y', '', 'full', 'any') and not ((search.ismatchscoring('x__NodeId:0', '', 'full', 'any') or search.ismatchscoring('x__NodeId:1391', '', 'full', 'any') or search.ismatchscoring('x__NodeId:8254', '', 'full', 'any') or search.ismatchscoring('x__NodeId:8255', '', 'full', 'any') or search.ismatchscoring('x__NodeId:8256', '', 'full', 'any') or search.ismatchscoring('x__NodeId:8257', '', 'full', 'any'))) and (((sortableDate ge 0 and sortableDate le 638550731349656733))) and search.ismatchscoring('spath:1273', '', 'full', 'any'))", the index may be offline, check exception information for details.

[Shazwazza]

Thanks, I'll continue to look into this tomorrow and see if I can replicate based on this info

[bielu]

@Shazwazza client agreed for me to share full umbraco log file, if you send give me email i will share it with you.

[Shazwazza]

Ok thx.

support at sdkits dot com

Will work.

[bielu]

@Shazwazza shared file with you, let me know if share worked for you

[Shazwazza]

Thanks @bielu . I've gone through all of the code changes since 6.0.1 and there isn't anything that has been modified to make this stop working after 6.0.1. I've gone through the logs and the query error and rebuilt what would be close to the output using the fluent api. I have managed to replicate the query parsing error, however, I've also reverted back to the 6.0.1 codebase with the same query and it produces the same query parsing error as well so I don't believe this is a regression from 6.0.1.

After some further analysis, this seems to be directly related to the path query that you are using: __Path:-20

In my tests, when I remove this part of the query, the query succeeds and parsing is successful. I will look further into this to see how to resolve this. In the meantime, can you specify how you are creating this specific path query? Is it just this .Not().Field("__Path", "-20") ? I just want to be sure that I'm replicating it the same way that you are querying.

[Shazwazza]

@bielu I've determined the cause - characters that need to be escaped are only processed when range queries are not detected. When range queries are detected, the query needs to be rewritten in OData format as you have seen. So the fix is to ensure that required chars are always escaped regardless of whether it is rewritten to OData or not.

I'll get a new release out shortly.

[bielu]

@Shazwazza that is interesting! and yes it is just .Not().Field("__Path", "-20"), we added it around time when we updated to 6.0.5 but we have similar query but not on this field for long time and it didnt caused issues! so it is really interesting that it caused issue.

[Shazwazza]

@bielu also note that for the Path:-20 query, I assume you are wanting to ignore everything underneath that path and not that specific path? If so, you would want to do this: `.Not().Field("Path", "-20,".MultipleCharacterWildcard())`

[Shazwazza]

This is fixed now https://github.com/SDKits/ExamineX/releases/tag/v6.0.6