NVIDIA / spark-rapids

Spark RAPIDS plugin - accelerate Apache Spark with GPUs
https://nvidia.github.io/spark-rapids
Apache License 2.0
822 stars 235 forks source link

[FEA] enable from_json and json scan by default #11630

Open revans2 opened 1 month ago

revans2 commented 1 month ago

Is your feature request related to a problem? Please describe. We will not be perfect in everything that we do, but we should be close enough that that we can enable most of from_json and json scan by default. There are still outstanding issues that prevent us from enabling all config options and all data types in all versions of Spark. There are also some odd corner cases that we need to at least document how we are different. We should also provide some configs to enable and disable data types individually.

Things that must be fixed before we can enable from_json and json scan by default

Types that we do not support by default:

Configs that we do not support by default:

Things that must be documented as corner cases: