Process spark query once in sql repo for handling validations and other operations.

opensearch-project / sql

Query your data using familiar SQL or intuitive Piped Processing Language (PPL)

https://opensearch.org/docs/latest/search-plugins/sql/index/

Apache License 2.0

116 stars 134 forks source link

Process spark query once in sql repo for handling validations and other operations. #2886

Open vamsi-amazon opened 1 month ago

vamsi-amazon commented 1 month ago

More details, comments in this PR: https://github.com/opensearch-project/sql/pull/2884/files Today, we might be running spark query through parser multiple times, ideally we should be parsing it once. Analyze and optimize these operations.

dblock commented 3 weeks ago

Catch All Triage - 1, 2, 3