Open davismcphee opened 5 months ago
Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)
Align the Discover behaviour with the CSV export. This would resolve the inconsistency, but if we go this route we should make sure flattened fields in general support adding leaf values as columns, otherwise the conflicting field type would have better support for leaf values than flattened fields directly.
Thx for raising this, I think your 2nd suggestion would be the best one in terms of UX. I've been working with flattened fields recently, and felt the need to select leaf nodes. However in terms of effort I guess it's not easy to do, since those leaf nodes are not returned by field_caps. What would be possible is to analyze the returned result, extract flattened fields subfields, and add those to a new subfield section of the field popover, so it's selectable. However this sounds simple, but ain't simple high likely.
But I would like this functionality!
Or we just add flattened fields support to ES|QL :)
Or we just add flattened fields support to ES|QL :)
Lower-hanging fruit than nested fields I believe :)
Or we just add flattened fields support to ES|QL :)
Lower-hanging fruit than nested fields I believe :)
@drewdaemon where's the code that we can hack it in? :)
Low hanging... but possibly still out of my reach haha. But hey, if we submit a PR, I'm sure the team would be happy to review :)
Summary
This issue arises when dealing with a data view that has a conflicting field of types
object
andflattened
.Normally in Discover, it's only possible to add the entire
flattened
field as a column, but not its properties since they aren't indexed as separate fields:But it's the opposite for
object
fields, where each property is flattened and indexed separately, so only the leaf values can be added as columns:When there's a conflict like this, Discover doesn't know which behaviour is correct, so it highlights the field as conflicting in the field list and allows adding both the root and leaf values (which it labels as multi fields) as columns:
But the way it handles them in the grid is different between documents. For the document where
user
is aflattened
field, it displays the full object for the root value column and empty for the leaf value columns. For the document whereuser
is anobject
field, it displays empty for the root value column and the property values for leaf value columns:Even more confusing is that when exporting a CSV with the
user.first
anduser.last
columns in the table, the CSV includes values for both the document with theflattened
field and the document with theobject
field:Solution
It's not clear which behaviour is correct or incorrect in this situation or how to resolve it, but there seem to be a few options:
flattened
fields in general support adding leaf values as columns, otherwise the conflicting field type would have better support for leaf values thanflattened
fields directly.Reproduction steps
The above scenario can be reproduced by running the following commands in console, creating a
flat-conflict-*
data view, and navigating to Discover: