elastic / elasticsearch

Free and Open Source, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
69.62k stars 24.64k forks source link

ESQL: Ensure data types in field attributes are always widened #112691

Open alex-spies opened 1 week ago

alex-spies commented 1 week ago

https://github.com/elastic/elasticsearch/pull/112610 needs to address multiple places where "raw" Elasticsearch data types somehow proliferate into the planner/optimizer; this led to bugs because ESQL only cares about a subset of data types that actually occur within the compute engine. Small numeric types like short and float are widened into compute engine's types, like integer resp. double.

For the longest time, this was okay because there was only 1 place where we dealt with this difference: while processing an index' mapping, we decide if a type is supported, and if it needs widening, and stuff it into a FieldAttribute. This means, we had an implicit invariant: data types inside FieldAttributes are always widened, resp. FieldAttributes' types are always types supported by the compute engine.

Let's enforce this invariant, at least via asserts, and let's make sure that type widening happens in as few places as possible. (We could also separate compute engine types from ES types and have FieldAttributes use the former, but that probably has a much wider blast radius that's not worth it.)

elasticsearchmachine commented 1 week ago

Pinging @elastic/es-analytical-engine (Team:Analytics)