Open mythrocks opened 4 days ago
The "correct" solution here would be to shim the code that generates the exception, ideally in RapidsErrorUtils
.
The problem is that RapidsErrorUtils
underwent refactor, as part of #11414. That change has yet to be merged. Attempting to fix this simultaneously will lead to rework from conflicts.
I'm not inclined to fix this as part of addressing #11009. I will include this repro as part of #11009, with an xfail
.
Description On Spark 4.0, when ANSI mode is enabled, and a
DECIMAL(3,0)
column is cast to a lower width type (e.g.DECIMAL(1,0)
), the plugin's error message does not match the one from Apache Spark.On Spark:
On the Spark RAPIDS plugin:
Repro
Here is a minimal
pytest
repro:Expected behavior The overflow exception should match what is produced from Spark 4.
Misc Depends on #11414.