search

opensearch-project / opensearch-spark

Spark Accelerator framework ; It enables secondary indices to remote data stores.
Apache License 2.0
22 stars 33 forks source link

[BUG] Expand doesn't support alias in GA environment #910

Open qianheng-aws opened 1 week ago

qianheng-aws commented 1 week ago

What is the bug? These 2 queries got failure because of syntax error.

source = myglue_test.default.region | where 'EUROPE' | eval array=json_array(1, 2, 3, 4) | expand array as uid | fields uid, r_name

source = myglue_test.tpch_csv.orders | eval array=json_array(1, 2, 3) | expand array as uid | fields uid, o_comment

How can one reproduce the bug? Run the above queries in GA env.

What is the expected behavior? It should run successfully.

What is your host/environment?

Do you have any screenshots? If applicable, add screenshots to help explain your problem.

Do you have any additional context? Looks like this commit is missing in GA branch p1-antlr-grammar: https://github.com/opensearch-project/opensearch-spark/pull/868

So the current env doesn't support AS after EXPAND, see diff:

p1-antlr-grammar:
expandCommand
    : EXPAND fieldExpression
    ;

main: 
expandCommand
    : EXPAND fieldExpression (AS alias = qualifiedName)?
    ;