Data view field conflicts resolving by filtering data

Kibana version: 8.14.0

Describe the bug: I know that field conflicts are an issue and the data view is telling me about it. I like to use the generic data views such as logs-* which enable this problem.

What I would expect is if I have to data streams:

demo-kibana.log
demo-kibana.audit

where one maps the source.ip as ip and one maps it as keyword. That the data view: demo-kibana.* shows a conflict. That is working as expected. Now when I add a KQL and filter down using constant keyword: demo-kibana.log I would expect the conflict to go away and for Kibana to let me use the field. Because within the same dataset the field is mapped the same. There is no conflict within demo-kibana.log there is a conflict between kibana.log and kibana.audit The issue is that this also forbids me to use in controls since the data view “errors”. I would expect Kibana to reevaluate the fields once I add a KQL that filters me away entire shards that create this issue, such as using constant keyword for data_stream.dataset.

Steps to reproduce:

PUT _index_template/demo-kibana.audit
{
  "template": {
    "mappings": {
      "properties": {
        "data_stream": {
          "type": "object",
          "properties": {
            "dataset": {
              "type": "constant_keyword"
            }
          }
        },
        "source": {
          "type": "object",
          "properties": {
            "ip": {
              "type": "keyword"
            }
          }
        }
      }
    }
  },
  "index_patterns": [
    "demo-kibana.audit-*"
  ],
  "data_stream": {}
}

PUT _index_template/demo-kibana.log
{
  "template": {
    "mappings": {
      "properties": {
        "data_stream": {
          "type": "object",
          "properties": {
            "dataset": {
              "type": "constant_keyword"
            }
          }
        },
        "source": {
          "type": "object",
          "properties": {
            "ip": {
              "type": "ip"
            }
          }
        }
      }
    }
  },
  "index_patterns": [
    "demo-kibana.log-*"
  ],
  "data_stream": {}
}

PUT _data_stream/demo-kibana.log-default
PUT _data_stream/demo-kibana.audit-default

POST demo-kibana.log-default/_doc
{
  "@timestamp": "2024-07-09T12:00:00.000Z",
  "data_stream": {
    "dataset": "kibana.log"
  },
  "source": {
    "ip": "192.168.0.1"
  }
}

POST demo-kibana.audit-default/_doc
{
  "@timestamp": "2024-07-09T12:00:00.000Z",
  "data_stream": {
    "dataset": "kibana.audit"
  },
  "source": {
    "ip": "192.168.0.1"
  }
}

Expected behavior:

When filtering down to a subset the fields shold be re-evaluated. Making sure that there are only conflicts within the actual queried indices.

Screenshots (if relevant):

elastic / kibana

Data view field conflicts resolving by filtering data #187860