search

opensearch-project / OpenSearch

🔎 Open source distributed and RESTful search engine.
https://opensearch.org/docs/latest/opensearch/index/
Apache License 2.0
9.77k stars 1.82k forks source link

[BUG] Constant score query fails to execute against a float field with "doc_values": false #12432

Closed marko-bekhta closed 8 months ago

marko-bekhta commented 8 months ago

Describe the bug

Trying to execute a constant score query with a simple term filter on a float field leads to an exception.

Related component

Search

To Reproduce

  1. Create a simple index with a float field and set "doc_values": false: 
    
PUT http://localhost:9200/test-float-index-000001
Content-Type: application/json

{ "mappings": { "dynamic": "strict", "properties": { "field0_Float": { "type": "float", "doc_values": false } } } }

2. Add some documents to the index:
```http
POST http://localhost:9200/test-float-index-000001/_doc/1
Content-Type: application/json

{
  "field0_Float": 123.12312
}
  1. Try to execute a constant_score search query: 
    
POST http://localhost:9200/test-float-index-000001/_search
Content-Type: application/json

{ "query": { "constant_score": { "filter": { "terms": { "field0_Float": [ 123.12312 ] } } } } }

4. Results in:
```json
{
  "error": {
    "root_cause": [
      {
        "type": "illegal_state_exception",
        "reason": "unexpected docvalues type NONE for field 'field0_Float' (expected one of [SORTED_NUMERIC, NUMERIC]). Re-index with correct docvalues type."
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "query",
    "grouped": true,
    "failed_shards": [
      {
        "shard": 0,
        "index": "test-float-index-000001",
        "node": "KVxFfRZ6S6W5DORNcyZmig",
        "reason": {
          "type": "illegal_state_exception",
          "reason": "unexpected docvalues type NONE for field 'field0_Float' (expected one of [SORTED_NUMERIC, NUMERIC]). Re-index with correct docvalues type."
        }
      }
    ],
    "caused_by": {
      "type": "illegal_state_exception",
      "reason": "unexpected docvalues type NONE for field 'field0_Float' (expected one of [SORTED_NUMERIC, NUMERIC]). Re-index with correct docvalues type.",
      "caused_by": {
        "type": "illegal_state_exception",
        "reason": "unexpected docvalues type NONE for field 'field0_Float' (expected one of [SORTED_NUMERIC, NUMERIC]). Re-index with correct docvalues type."
      }
    }
  },
  "status": 500
}
org.opensearch.action.search.SearchPhaseExecutionException: all shards failed
    at org.opensearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:722) [opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:379) [opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:761) [opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.action.search.AbstractSearchAsyncAction.onShardFailure(AbstractSearchAsyncAction.java:513) [opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.action.search.AbstractSearchAsyncAction$1.onFailure(AbstractSearchAsyncAction.java:301) [opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.action.search.SearchExecutionStatsCollector.onFailure(SearchExecutionStatsCollector.java:104) [opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:75) [opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.action.search.SearchTransportService$ConnectionCountingHandler.handleException(SearchTransportService.java:766) [opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.transport.TransportService$9.handleException(TransportService.java:1725) [opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1511) [opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.transport.TransportService$DirectResponseChannel.processException(TransportService.java:1625) [opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1599) [opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.transport.TaskTransportChannel.sendResponse(TaskTransportChannel.java:81) [opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.transport.TransportChannel.sendErrorResponse(TransportChannel.java:73) [opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.action.support.ChannelActionListener.onFailure(ChannelActionListener.java:70) [opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.action.ActionRunnable.onFailure(ActionRunnable.java:104) [opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:54) [opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.threadpool.TaskAwareRunnable.doRun(TaskAwareRunnable.java:78) [opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) [opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:59) [opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:913) [opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) [opensearch-2.12.0.jar:2.12.0]
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) [?:?]
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) [?:?]
    at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
Caused by: org.opensearch.OpenSearchException$3: unexpected docvalues type NONE for field 'field0_Float' (expected one of [SORTED_NUMERIC, NUMERIC]). Re-index with correct docvalues type.
    at org.opensearch.OpenSearchException.guessRootCauses(OpenSearchException.java:708) ~[opensearch-core-2.12.0.jar:2.12.0]
    at org.opensearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:377) [opensearch-2.12.0.jar:2.12.0]
    ... 23 more
Caused by: java.lang.IllegalStateException: unexpected docvalues type NONE for field 'field0_Float' (expected one of [SORTED_NUMERIC, NUMERIC]). Re-index with correct docvalues type.
    at org.apache.lucene.index.DocValues.checkField(DocValues.java:218) ~[lucene-core-9.9.2.jar:9.9.2 a2939784c4ca60bc28bf488b5479c02fc2e5e22c - 2024-01-25 09:51:09]
    at org.apache.lucene.index.DocValues.getSortedNumeric(DocValues.java:299) ~[lucene-core-9.9.2.jar:9.9.2 a2939784c4ca60bc28bf488b5479c02fc2e5e22c - 2024-01-25 09:51:09]
    at org.apache.lucene.document.SortedNumericDocValuesSetQuery$1.scorer(SortedNumericDocValuesSetQuery.java:109) ~[lucene-core-9.9.2.jar:9.9.2 a2939784c4ca60bc28bf488b5479c02fc2e5e22c - 2024-01-25 09:51:09]
    at org.apache.lucene.search.Weight.scorerSupplier(Weight.java:135) ~[lucene-core-9.9.2.jar:9.9.2 a2939784c4ca60bc28bf488b5479c02fc2e5e22c - 2024-01-25 09:51:09]
    at org.apache.lucene.search.IndexOrDocValuesQuery$1.scorerSupplier(IndexOrDocValuesQuery.java:170) ~[lucene-core-9.9.2.jar:9.9.2 a2939784c4ca60bc28bf488b5479c02fc2e5e22c - 2024-01-25 09:51:09]
    at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.scorerSupplier(LRUQueryCache.java:737) ~[lucene-core-9.9.2.jar:9.9.2 a2939784c4ca60bc28bf488b5479c02fc2e5e22c - 2024-01-25 09:51:09]
    at org.opensearch.indices.IndicesQueryCache$CachingWeightWrapper.scorerSupplier(IndicesQueryCache.java:182) ~[opensearch-2.12.0.jar:2.12.0]
    at org.apache.lucene.search.ConstantScoreQuery$1.scorerSupplier(ConstantScoreQuery.java:153) ~[lucene-core-9.9.2.jar:9.9.2 a2939784c4ca60bc28bf488b5479c02fc2e5e22c - 2024-01-25 09:51:09]
    at org.apache.lucene.search.Weight.bulkScorer(Weight.java:165) ~[lucene-core-9.9.2.jar:9.9.2 a2939784c4ca60bc28bf488b5479c02fc2e5e22c - 2024-01-25 09:51:09]
    at org.apache.lucene.search.ConstantScoreQuery$1.bulkScorer(ConstantScoreQuery.java:142) ~[lucene-core-9.9.2.jar:9.9.2 a2939784c4ca60bc28bf488b5479c02fc2e5e22c - 2024-01-25 09:51:09]
    at org.opensearch.search.internal.ContextIndexSearcher$1.bulkScorer(ContextIndexSearcher.java:383) ~[opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.search.internal.ContextIndexSearcher.searchLeaf(ContextIndexSearcher.java:324) ~[opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:283) ~[opensearch-2.12.0.jar:2.12.0]
    at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:554) ~[lucene-core-9.9.2.jar:9.9.2 a2939784c4ca60bc28bf488b5479c02fc2e5e22c - 2024-01-25 09:51:09]
    at org.opensearch.search.query.QueryPhase.searchWithCollector(QueryPhase.java:356) ~[opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.search.query.QueryPhase$DefaultQueryPhaseSearcher.searchWithCollector(QueryPhase.java:443) ~[opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.search.query.QueryPhase$DefaultQueryPhaseSearcher.searchWith(QueryPhase.java:427) ~[opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.search.query.QueryPhaseSearcherWrapper.searchWith(QueryPhaseSearcherWrapper.java:60) ~[opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.neuralsearch.search.query.HybridQueryPhaseSearcher.searchWith(HybridQueryPhaseSearcher.java:74) ~[?:?]
    at org.opensearch.search.query.QueryPhase.executeInternal(QueryPhase.java:282) ~[opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.search.query.QueryPhase.execute(QueryPhase.java:155) ~[opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:547) ~[opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.search.SearchService.executeQueryPhase(SearchService.java:611) ~[opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.search.SearchService$2.lambda$onResponse$0(SearchService.java:580) ~[opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.action.ActionRunnable.lambda$supply$0(ActionRunnable.java:74) ~[opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.action.ActionRunnable$2.doRun(ActionRunnable.java:89) ~[opensearch-2.12.0.jar:2.12.0]
    at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) ~[opensearch-2.12.0.jar:2.12.0]
    ... 8 more

If a simple term query is executed:

POST http://localhost:9200/test-float-index-000001/_search
Content-Type: application/json

{
  "query": {
    "terms": {
      "field0_Float": [
        123.12312
      ]
    }
  }
}

or if doc values are set to true ("doc_values": true), then there's no error.

This bug was introduced in 2.12.0; it is not reproducible in previous 2.11.1 and earlier versions.

Expected behavior

It is expected that a constant score query in this case (with simple term filter on a float field) does not fail and returns the matched documents.

Additional Details

Plugins eabfec3b01ea opensearch-alerting 2.12.0.0 eabfec3b01ea opensearch-anomaly-detection 2.12.0.0 eabfec3b01ea opensearch-asynchronous-search 2.12.0.0 eabfec3b01ea opensearch-cross-cluster-replication 2.12.0.0 eabfec3b01ea opensearch-custom-codecs 2.12.0.0 eabfec3b01ea opensearch-flow-framework 2.12.0.0 eabfec3b01ea opensearch-geospatial 2.12.0.0 eabfec3b01ea opensearch-index-management 2.12.0.0 eabfec3b01ea opensearch-job-scheduler 2.12.0.0 eabfec3b01ea opensearch-knn 2.12.0.0 eabfec3b01ea opensearch-ml 2.12.0.0 eabfec3b01ea opensearch-neural-search 2.12.0.0 eabfec3b01ea opensearch-notifications 2.12.0.0 eabfec3b01ea opensearch-notifications-core 2.12.0.0 eabfec3b01ea opensearch-observability 2.12.0.0 eabfec3b01ea opensearch-performance-analyzer 2.12.0.0 eabfec3b01ea opensearch-reports-scheduler 2.12.0.0 eabfec3b01ea opensearch-security 2.12.0.0 eabfec3b01ea opensearch-security-analytics 2.12.0.0 eabfec3b01ea opensearch-skills 2.12.0.0 eabfec3b01ea opensearch-sql 2.12.0.0

Host/Environment (please complete the following information): Linux Fedora 6.7.4-200.fc39.x86_64

Additional context OpenSearch is started from an official docker image as:

docker run -it -p 9200:9200 -p 9600:9600 -e "discovery.type=single-node" -e "plugins.security.disabled=true" -e "OPENSEARCH_INITIAL_ADMIN_PASSWORD=Actua11yStrongPassword" opensearchproject/opensearch:2.12.0
peternied commented 8 months ago

[Triage - attendees 1 2 3 4 5] @marko-bekhta Thanks for filing this issue, we would glady review a pull request for this issue.