search

NVIDIA / spark-rapids

Spark RAPIDS plugin - accelerate Apache Spark with GPUs
https://nvidia.github.io/spark-rapids
Apache License 2.0
788 stars 228 forks source link

[FEA] Support CharType and VarcharType in ParquetCachedBatchSerializer #1225

Open jlowe opened 3 years ago

jlowe commented 3 years ago

Spark's HiveStringType was recently removed, with the functionality apparently covered by CharType and VarcharType. These types should be supported in the cached batch serializer.

sameerz commented 3 years ago

Need to investigate ParquetCachedBatchSerializer and other processing whether the new types similar to string types need to be updated in the plugin.