search

NVIDIA / spark-rapids

Spark RAPIDS plugin - accelerate Apache Spark with GPUs
https://nvidia.github.io/spark-rapids
Apache License 2.0
805 stars 234 forks source link

[FEA] Support StaticInvoke to convert from string to varchar #11233

Open viadea opened 3 months ago

viadea commented 3 months ago

I wish we can support StaticInvoke to convert from string to varchar.

eg.

create table src(id string) stored as parquet;
create table tgt(id varchar(200)) stored as parquet ;
insert into src values ('abc');
insert into tgt select id from src;

Fallback message:

      ! <StaticInvoke> staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, varcharTypeWriteSideCheck, cast(id#25 as string), 200, true, false, true) cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke
revans2 commented 3 months ago

There are a lot of things about this that are new. We have never done anything with StaticInvoke before. We also do not have support for the varchar data type. is not something that we have supported before. For this very specific case it looks do-able but because we have not done a number of these things before I want to be a bit cautious about that.

Please note that StaticInvoke is a very special operator and we will not be able to support it generally.

gerashegalov commented 3 months ago

Related to https://github.com/NVIDIA/spark-rapids/issues/8324