search

apache / pinot

Apache Pinot - A realtime distributed OLAP datastore
https://pinot.apache.org/
Apache License 2.0
5.31k stars 1.24k forks source link

ScalarTransformFunctionWrapper throws exception for Object return type #11743

Open abhioncbr opened 10 months ago

abhioncbr commented 10 months ago

ScalarTransformFunctionWrapper throws an exception for any transformation function with Object return type. for example, casewhen or cast

Here is one such stack trace:

java.lang.IllegalStateException: Cannot convert ColumnDataType: OBJECT to DataType

    at org.apache.pinot.common.utils.DataSchema$ColumnDataType.toDataType(DataSchema.java:311)
    at org.apache.pinot.core.operator.transform.function.ScalarTransformFunctionWrapper.<init>(ScalarTransformFunctionWrapper.java:69)
    at org.apache.pinot.core.operator.transform.function.TransformFunctionFactory.get(TransformFunctionFactory.java:316)
    at org.apache.pinot.core.operator.transform.function.TransformFunctionFactory.get(TransformFunctionFactory.java:347)

this was introduced with the PR, specific code change

Jackie-Jiang commented 10 months ago

Good catch!

IMO we should not include ColumnDataType.OBJECT into FunctionUtils.COLUMN_DATA_TYPE_MAP (introduced in #8608) because OBJECT is just a placeholder column data type for intermediate aggregation result, and should not be exposed to the data schema. @walterddr Can you also double check if we can remove it?

walterddr commented 10 months ago

good point. let me double check and see if the planner side have any side effect if we remove.