Improve JSON scan and `from_json` - Githubissues

NVIDIA / spark-rapids

Spark RAPIDS plugin - accelerate Apache Spark with GPUs

https://nvidia.github.io/spark-rapids

Apache License 2.0

822 stars 235 forks source link

Improve JSON scan and `from_json` #11702

Closed ttnghia closed 1 week ago

ttnghia commented 2 weeks ago

This implements a few improvements for JSON scan and from_json:

For both JSON scan and from_json: support disabling allowSingleQuotes option.
For from_json with map<string, string> schema:
- Add the ability to pass down options for turning on/off allowSingleQuotes, allowNumericLeadingZeros, allowNonNumericNumbers, and allowUnquotedControlChars, along with the corresponding tests.
- Add more tests.

Depends on:

https://github.com/NVIDIA/spark-rapids-jni/pull/2562

ttnghia commented 1 week ago

build