term condition does not work

[piemon-nyah]

Elasticsearch Version

7.12.0

Installed Plugins

No response

Java Version

bundled

OS Version

AnolisOS Linux 7.9

Problem Description

When I used this DSL query, I found that my query condition was invalid. It returned not only one piece of data, but also fields other than app_name and mdc, and data that was not cos-core.

GET log-pro-002846/_search
{
  "_source": ["app_name","mdc"], 
  "from": 0,
  "size": 10,
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "app_name": {
              "value": "cos-core"
            }
          }
        },
        {
          "term": {
            "mdc.x-cos-request-id": {
              "value": "84b0bb8dca944c48bb374ba01b6f3456"
            }
          }
        },
        {
        "range": {
          "@timestamp": {
              "from": "2024-08-09 00:00:18",
              "to": "2024-08-09 23:17:18",
              "include_lower": true,
              "include_upper": true,
              "time_zone": "Asia/Shanghai",
              "format": "yyyy-MM-dd HH:mm:ss"
          }
        }
      }
      ]}
  },
  "sort": [
    {
      "@timestamp": {
        "order": "desc"
      }
    }
  ]
}

However, when I removed the term condition of mdc.x-cos-request-id, the query was normal. I made sure that there is actually data such as mdc.x-cos-request-id equal to 84b0bb8dca944c48bb374ba01b6f3456 mapping is this

{
  "mappings": {
    "_doc": {
      "dynamic": "runtime",
      "runtime": {
        "mdc.x-cos-request-id": {
          "type": "keyword"
        }
      },
      "properties": {
        "@timestamp": {
          "type": "date"
        },
        "app_env": {
          "type": "keyword"
        },
        "app_loc": {
          "type": "keyword"
        },
        "app_name": {
          "type": "keyword"
        },
        "class": {
          "type": "keyword"
        },
        "exception": {
          "properties": {
            "exception_class": {
              "type": "keyword"
            },
            "exception_message": {
              "type": "text"
            },
            "stacktrace": {
              "type": "text",
              "copy_to": [
                "full_message_stacktrace"
              ]
            }
          }
        },
        "file": {
          "type": "keyword"
        },
        "full_message_stacktrace": {
          "type": "text"
        },
        "level": {
          "type": "keyword"
        },
        "line_number": {
          "type": "integer"
        },
        "logger_name": {
          "type": "keyword"
        },
        "mdc": {
          "type": "object"
        },
        "message": {
          "type": "text",
          "copy_to": [
            "full_message_stacktrace"
          ]
        },
        "method": {
          "type": "keyword"
        },
        "source_host": {
          "type": "keyword"
        },
        "source_ip": {
          "type": "keyword"
        },
        "thread_name": {
          "type": "keyword"
        }
      }
    }
  }
}

Steps to Reproduce

As mentioned above

Logs (if relevant)

No response

[gbanasiak]

@piemon-nyah Can you reproduce this on the latest 7.17.x? See https://www.elastic.co/support/eol.

[elasticsearchmachine]

Pinging @elastic/es-search-relevance (Team:Search Relevance)

[benwtrent]

I suggest using the https://www.elastic.co/guide/en/elasticsearch/reference/current/search-validate.html API with the rewrite parameter to see what low level query your provided one gets transformed into.

Additionally, using the explain API: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-explain.html

Both of those should help debug this further & please try to reproduce on a modern version (at least latest 7 series). We can try to reproduce ourselves, but that makes it difficult without any test data.

Additionally, that type of runtime field is pulling directly from _source, so knowing how that value is included in source (if its mdc: {x-cos-request-id: ...} vs mdc.x-cos-request-id: ...)