search

elastic / elasticsearch

Free and Open Source, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
69.66k stars 24.65k forks source link

`keyword` field failure in synthetic source with `flattened` field #113538

Open salvatore-campagna opened 2 hours ago

salvatore-campagna commented 2 hours ago

Elasticsearch Version

all

Installed Plugins

No response

Java Version

bundled

OS Version

all

Problem Description

The yaml test includes a test where a keyword field is not reconstructed correctly in synthetic source. It looks like the issue happens only when a flattened field is mapped too. I could not reproduce the issue without the flattened field.

See there is a commented line in the test

  #TODO: synthetic source field reconstruction bug (TBD: add link to the issue here)
  #- match: { hits.hits.0._source.keyword: ["foo bar", "the quick brown fox"] }

Steps to Reproduce

Just execute the yaml test uncommenting the line under TODO.

Logs (if relevant)

No response

elasticsearchmachine commented 2 hours ago

Pinging @elastic/es-storage-engine (Team:StorageEngine)

lkts commented 1 hour ago

This is due to this line https://github.com/elastic/elasticsearch/blob/4f666310c7dff24d3f7704635a8faf01dcc140c7/server/src/main/java/org/elasticsearch/index/mapper/KeywordFieldMapper.java#L1096 Both fieldType().ignoreAbove and ignoreAboveDefault are 10 in this case.

lkts commented 1 hour ago

I think wildcard has the same problem.

lkts commented 1 hour ago

https://github.com/elastic/elasticsearch/blob/4f666310c7dff24d3f7704635a8faf01dcc140c7/x-pack/plugin/src/yamlRestTest/resources/rest-api-spec/test/wildcard/30_ignore_above_synthetic_source.yml#L52

this test is not correct, _source should include values that are ignored meaning "jumps over the lazy dog" should be present