Conflicting fields with object fields are wrongly detected as multi-fields

[timroes]

When having a conflicting field where a field is in one index a string type and in another index an object type Discover will no longer show the value in the expanded document for the documetns where it's an object type.

To reproduce execute the following commands in the dev tools:

PUT discover1
{
  "mappings": {
    "properties": {
      "field": {
        "type": "keyword"
      }
    }
  }
}

PUT discover2
{
  "mappings": {
    "properties": {
      "field": {
        "properties": {
          "subfield": {
            "type": "keyword"
          }
        }
      }
    }
  }
}

POST discover1/_doc
{
  "field": "a string directly"
}

POST discover2/_doc
{
  "field": {
    "subfield": "nested in an object"
  }
}

Now create an index pattern for discover*. When going to Discover you will see the documents as follows:

You can see that even though the field is correctly returned in the "summed up" _source/Document view as field.subfield that if you expand the document it's not listed in there. The behavior happens whether fields or _source mode is used.

Update: The actual problem here is that field.subfield is by the index pattern wrongly detected as a multi-field while it actually isn't a multi-field. Expected behavior would be to not wrongly detect field.subfield as a multi-field.

Tested on 7.14

[elasticmachine]

Pinging @elastic/kibana-app (Team:KibanaApp)

[majagrubic]

The above scenario is not related to the conflicting fields. field.subfield is no longer shown as it is a multi-field, and these are hidden by default. Enabling showMultiFields in the advanced settings will display the field:

[majagrubic]

As for reading from source bit, that is indeed a bug, as multifields should always be shown in that case.

[timroes]

You're right, the actual problem here is, that field.subfield is wrongly detected as a multi-field, while it isn't one. Since I believe the primary issue here of wrongly detecting multi-fields might fix that behavior for both implementations, let's use this issue to keep track of not detecting conflicting fields wrongly as multi-fields.

[elasticmachine]

Pinging @elastic/kibana-app-services (Team:AppServices)

[mattkime]

This bug exists because the field caps api doesn't report info about multi-fields, rather its inferred. This inference is incorrect in exactly the circumstance described here but there's nothing we can do until https://github.com/elastic/elasticsearch/issues/75474 is addressed.

[elasticmachine]

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

[kertal]

@mattkime it seems https://github.com/elastic/elasticsearch/issues/75474 is resolved, so this might be unblocked?

[mattkime]

@kertal I'd need to do a closer investigation. The solution doesn't match what was requested so its not clear to me what it would take to get the result we'd want.