prestodb / presto

The official home of the Presto distributed SQL query engine for big data
http://prestodb.io
Apache License 2.0
16.07k stars 5.38k forks source link

Add expression evaluation in Presto sidecar #24126

Open tdcmeehan opened 2 days ago

tdcmeehan commented 2 days ago

Description

To support constant folding and consistent semantics between the Presto coordinator (Java) and the Presto C++ worker, it is necessary to use consistent expression evaluation. To support this, a native expression evaluation endpoint has been added to the Presto sidecar process, and a plugin has been created which can utilize Velox expression evaluation behind a standard ExpressionOptimizer.

Motivation and Context

Consistency between C++ and Java semantics. Support for using C++ functions during constant folding of expressions in the planner.

Impact

Fix some lingering bugs with C++ clusters.

Test Plan

Tests have been added by extending the TestRowExpressionInterpreter class to also test native expression evaluation. However, this feature is still in Beta, and to support production workloads with complete certainty a fuzzer will be created to surface any remaining bugs with the integration at a later time.

Contributor checklist

Release Notes

Please follow release notes guidelines and fill in the release notes below.

== RELEASE NOTES ==

General Changes
* ... :pr:`12345`
* ... :pr:`12345`

Hive Connector Changes
* ... :pr:`12345`
* ... :pr:`12345`

If release note is NOT required, use:

== NO RELEASE NOTE ==