NVIDIA / spark-rapids-jni

RAPIDS Accelerator JNI For Apache Spark
Apache License 2.0
36 stars 64 forks source link

Disable large-string support in cudf configuration #2189

Closed jlowe closed 3 months ago

jlowe commented 3 months ago

Is your feature request related to a problem? Please describe. rapidsai/cudf#16037 will default libcudf to enable large string support. The RAPIDS Accelerator has many places in C++ and Java where it assumes the size of offsets is 4 bytes, and thus large strings would break in various ways. Until the RAPIDS Accelerator is ready for large string support, we will need to disable this in the cudf configuration.

Describe the solution you'd like libcudf build needs to configure with CUDF_LARGE_STRINGS_DISABLED