apache / datafusion

Apache DataFusion SQL Query Engine
https://datafusion.apache.org/
Apache License 2.0
5.87k stars 1.11k forks source link

Implement the rewrite from the Map literal to Map function #11434

Closed goldmedal closed 1 month ago

goldmedal commented 1 month ago

Is your feature request related to a problem or challenge?

Based on the discussion in https://github.com/apache/datafusion/issues/11268#issuecomment-2211125762, we will support the MAP literal to create a map value. Referring to the DuckDB syntax, we can use syntax like:

SELECT MAP {'key1': 10, 'key2': 20, 'key3': 30};

Describe the solution you'd like

After #11361, we now have two functions: map and make_map. We can use UserDefinedSQLPlanner to implement the proposed syntax and rewrite it to the map function. As @jayzhan211 mentioned in https://github.com/apache/datafusion/pull/11361#issuecomment-2221710127, we can also use ExprPlanner to handle the make_map function. Since make_map is less efficient than the map function, we should retain only the faster map scalar function implementation. For more details, see the previous discussions and benchmark results in #11361.

Describe alternatives you've considered

No response

Additional context

No response

goldmedal commented 1 month ago

take

goldmedal commented 1 month ago

I found the MAP literal isn't supported by the sqlparser-rs. I think we need to implement this syntax on the sqlparser-rs side first. However, I think I can have a PR for moving make_map to ExprPlanner first. What do you think? @jayzhan211

goldmedal commented 1 month ago

The part of moving MAKE_MAP has been solved by #11452. I'll start to implement the MAP literal after https://github.com/sqlparser-rs/sqlparser-rs/pull/1344