search

NVIDIA / spark-rapids

Spark RAPIDS plugin - accelerate Apache Spark with GPUs
https://nvidia.github.io/spark-rapids
Apache License 2.0
789 stars 228 forks source link

[BUG] Scan and JsonToStruct shoudl invalidate a struct in an array on out of bounds #11510

Closed revans2 closed 3 hours ago

revans2 commented 3 hours ago

Describe the bug This is very similar to https://github.com/NVIDIA/spark-rapids/issues/11491 but happens for items in an struct if the struct is in an array, and only if it is for 3.4.1 or below.

I went looking for which issue/PR fixed this, but there was no obvious smoking gun here.

Steps/Code to reproduce bug Run the integration tests that are marked as xfail with this issue pointing to them.

revans2 commented 3 hours ago

Oops this is a dupe of https://github.com/NVIDIA/spark-rapids/issues/10588