Closed cristian-marisescu closed 2 weeks ago
Size is for array_length: https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.functions.size.html
The function you are wanting is length: https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.functions.length.html
Hi,
F.size is failing due to wrong selected function
Code
Error
generated SQL on size_test.sql():
Seems like wrong sql is generated. I think it should be LENGTH, not ARRAY_LENGTH. Running the sql with LENGTH works in DuckDB.