Open goldmedal opened 4 days ago
I think maybe you can enable array_expressions feature? https://github.com/apache/datafusion#crate-features
I think maybe you can enable array_expressions feature? https://github.com/apache/datafusion#crate-features
Thanks @yyy1000
I tried to enable the feature in 39.0.0
and the nightly version.
datafusion = {git = "https://github.com/apache/datafusion", branch = "main", features = ["array_expressions"]}
and
datafusion = {"version" = "39.0.0", features = ["array_expressions"]}
However, it doesn't work. I think array_expressions
is for the array function, not the logical plan for the array value.
Is your feature request related to a problem or challenge?
In
unparser/expr.rs
, list scalar values haven't been supported yet.However, I found some problems with supporting it. Currently, array construction is implemented by a
ScalarUDF
calledmake_array
. https://github.com/apache/datafusion/blob/ff116c3da69897358f210a3ea944c8e51dcb7b52/datafusion/sql/src/expr/value.rs#L145-L151 This might meanScalarValue::List
is never created from an AST expression node.If I try to create the logical plan without the default SessionContext, like this:
I will get the error message:
Then, I tried to add
make_array_udf
inMyContextProvider
. I can get the unparsed result like this:However, I think
make_array
isn't a common function for standard SQL. It may cause some problems when we try to fully push down the unparsed result to a specific data source.Describe the solution you'd like
Ideally, we can round-trip the array construction in SQL
select [1,2,3]
through the following steps:AST::Expr::Array
toScalarValue::List
(currently, it's themake_array
scalar function).ScalarValue::List
.However, I'm not sure why the array expression was disabled and
make_array
is used instead. Maybe it is due to some performance issues?Describe alternatives you've considered
Another approach is to try unparsing the
make_array()
plan toAST::Expr::Array
. It's tricky, but I think it could work.Additional context
No response