search

apache / incubator-gluten

Gluten is a middle layer responsible for offloading JVM-based SQL engines' execution to native engines.
https://gluten.apache.org/
Apache License 2.0
1.13k stars 409 forks source link

[VL] Cast function's support status #5936

Open PHILO-HE opened 2 months ago

PHILO-HE commented 2 months ago

Description

Not any pair of types can be converted from each other. Spark has a check here. Gluten cpp has some pre-check logic for cast expression. See Gluten code here.

This is the output from a tool proposed by https://github.com/apache/incubator-gluten/pull/6392 ("!!" marks not supported cast case).

## cast validation passes: cast from BooleanType to ByteType
## cast validation passes: cast from BooleanType to ShortType
## cast validation passes: cast from BooleanType to IntegerType
## cast validation passes: cast from BooleanType to LongType
## cast validation passes: cast from BooleanType to FloatType
## cast validation passes: cast from BooleanType to DoubleType
## cast validation passes: cast from BooleanType to DecimalType(5,1)
## cast validation passes: cast from BooleanType to StringType
!! cast validation fails: cast from BooleanType to TimestampType

## cast validation passes: cast from ByteType to BooleanType
## cast validation passes: cast from ByteType to ShortType
## cast validation passes: cast from ByteType to IntegerType
## cast validation passes: cast from ByteType to LongType
## cast validation passes: cast from ByteType to FloatType
## cast validation passes: cast from ByteType to DoubleType
## cast validation passes: cast from ByteType to DecimalType(5,1)
## cast validation passes: cast from ByteType to StringType
## cast validation passes: cast from ByteType to BinaryType
!! cast validation fails: cast from ByteType to TimestampType

## cast validation passes: cast from ShortType to BooleanType
## cast validation passes: cast from ShortType to ByteType
## cast validation passes: cast from ShortType to IntegerType
## cast validation passes: cast from ShortType to LongType
## cast validation passes: cast from ShortType to FloatType
## cast validation passes: cast from ShortType to DoubleType
## cast validation passes: cast from ShortType to DecimalType(5,1)
## cast validation passes: cast from ShortType to StringType
## cast validation passes: cast from ShortType to BinaryType
!! cast validation fails: cast from ShortType to TimestampType

## cast validation passes: cast from IntegerType to BooleanType
## cast validation passes: cast from IntegerType to ByteType
## cast validation passes: cast from IntegerType to ShortType
## cast validation passes: cast from IntegerType to LongType
## cast validation passes: cast from IntegerType to FloatType
## cast validation passes: cast from IntegerType to DoubleType
## cast validation passes: cast from IntegerType to DecimalType(5,1)
## cast validation passes: cast from IntegerType to StringType
## cast validation passes: cast from IntegerType to BinaryType
!! cast validation fails: cast from IntegerType to TimestampType

## cast validation passes: cast from LongType to BooleanType
## cast validation passes: cast from LongType to ByteType
## cast validation passes: cast from LongType to ShortType
## cast validation passes: cast from LongType to IntegerType
## cast validation passes: cast from LongType to FloatType
## cast validation passes: cast from LongType to DoubleType
## cast validation passes: cast from LongType to DecimalType(5,1)
## cast validation passes: cast from LongType to StringType
## cast validation passes: cast from LongType to BinaryType
!! cast validation fails: cast from LongType to TimestampType

## cast validation passes: cast from FloatType to BooleanType
## cast validation passes: cast from FloatType to ByteType
## cast validation passes: cast from FloatType to ShortType
## cast validation passes: cast from FloatType to IntegerType
## cast validation passes: cast from FloatType to LongType
## cast validation passes: cast from FloatType to DoubleType
## cast validation passes: cast from FloatType to DecimalType(5,1)
## cast validation passes: cast from FloatType to StringType
!! cast validation fails: cast from FloatType to TimestampType

## cast validation passes: cast from DoubleType to BooleanType
## cast validation passes: cast from DoubleType to ByteType
## cast validation passes: cast from DoubleType to ShortType
## cast validation passes: cast from DoubleType to IntegerType
## cast validation passes: cast from DoubleType to LongType
## cast validation passes: cast from DoubleType to FloatType
## cast validation passes: cast from DoubleType to DecimalType(5,1)
## cast validation passes: cast from DoubleType to StringType
!! cast validation fails: cast from DoubleType to TimestampType

## cast validation passes: cast from DecimalType(5,1) to BooleanType
## cast validation passes: cast from DecimalType(5,1) to ByteType
## cast validation passes: cast from DecimalType(5,1) to ShortType
## cast validation passes: cast from DecimalType(5,1) to IntegerType
## cast validation passes: cast from DecimalType(5,1) to LongType
## cast validation passes: cast from DecimalType(5,1) to FloatType
## cast validation passes: cast from DecimalType(5,1) to DoubleType
## cast validation passes: cast from DecimalType(5,1) to StringType
!! cast validation fails: cast from DecimalType(5,1) to TimestampType

## cast validation passes: cast from StringType to BooleanType
## cast validation passes: cast from StringType to ByteType
## cast validation passes: cast from StringType to ShortType
## cast validation passes: cast from StringType to IntegerType
## cast validation passes: cast from StringType to LongType
## cast validation passes: cast from StringType to FloatType
## cast validation passes: cast from StringType to DoubleType
## cast validation passes: cast from StringType to DecimalType(5,1)
## cast validation passes: cast from StringType to BinaryType
!! cast validation fails: cast from StringType to DateType
!! cast validation fails: cast from StringType to TimestampType

## cast validation passes: cast from BinaryType to BooleanType
## cast validation passes: cast from BinaryType to ByteType
## cast validation passes: cast from BinaryType to ShortType
## cast validation passes: cast from BinaryType to IntegerType
## cast validation passes: cast from BinaryType to LongType
## cast validation passes: cast from BinaryType to FloatType
## cast validation passes: cast from BinaryType to DoubleType
## cast validation passes: cast from BinaryType to DecimalType(5,1)
## cast validation passes: cast from BinaryType to StringType
!! cast validation fails: cast from BinaryType to DateType
!! cast validation fails: cast from BinaryType to TimestampType

!! cast validation fails: cast from DateType to BooleanType
!! cast validation fails: cast from DateType to ByteType
!! cast validation fails: cast from DateType to ShortType
!! cast validation fails: cast from DateType to IntegerType
!! cast validation fails: cast from DateType to LongType
!! cast validation fails: cast from DateType to FloatType
!! cast validation fails: cast from DateType to DoubleType
!! cast validation fails: cast from DateType to DecimalType(5,1)
!! cast validation fails: cast from DateType to StringType
!! cast validation fails: cast from DateType to TimestampType

!! cast validation fails: cast from TimestampType to BooleanType
!! cast validation fails: cast from TimestampType to ByteType
!! cast validation fails: cast from TimestampType to ShortType
!! cast validation fails: cast from TimestampType to IntegerType
!! cast validation fails: cast from TimestampType to LongType
!! cast validation fails: cast from TimestampType to FloatType
!! cast validation fails: cast from TimestampType to DoubleType
!! cast validation fails: cast from TimestampType to DecimalType(5,1)
!! cast validation fails: cast from TimestampType to StringType
!! cast validation fails: cast from TimestampType to DateType

## cast validation passes: cast from NullType to BooleanType
## cast validation passes: cast from NullType to ByteType
## cast validation passes: cast from NullType to ShortType
## cast validation passes: cast from NullType to IntegerType
## cast validation passes: cast from NullType to LongType
## cast validation passes: cast from NullType to FloatType
## cast validation passes: cast from NullType to DoubleType
## cast validation passes: cast from NullType to DecimalType(5,1)
## cast validation passes: cast from NullType to StringType
## cast validation passes: cast from NullType to BinaryType
## cast validation passes: cast from NullType to DateType
!! cast validation fails: cast from NullType to TimestampType
## cast validation passes: cast from NullType to ArrayType(IntegerType,true)
## cast validation passes: cast from NullType to MapType(StringType,IntegerType,true)
## cast validation passes: cast from NullType to StructType(StructField(a,StringType,true), StructField(b,IntegerType,true))

!! cast validation fails: cast from ArrayType(IntegerType,true) to StringType
!! cast validation fails: cast from MapType(StringType,IntegerType,true) to StringType
!! cast validation fails: cast from StructType(StructField(a,StringType,true), StructField(b,IntegerType,true)) to StringType
FelixYBW commented 2 months ago

We do have the table to support cast between data types but not fully tested. The goal of this issue is to track the support and test status of all cast cases.

@zhli1142015 @Yohahaha @ulysses-you @jackylee-ch @kecookier @WangGuangxin

PHILO-HE commented 1 month ago

Just updated the list.