add support for spark 3.1.1(emr 6.3.0)

aws / sagemaker-spark-container

The SageMaker Spark Container is a Docker image used to run data processing workloads with the Spark framework on Amazon SageMaker.

Apache License 2.0

36 stars 74 forks source link

Closed guoqiao1992 closed 3 years ago

guoqiao1992 commented 3 years ago

Issue #, if available: Spark container only supports spark 3.0.0 today. Need to support 3.1.1 due to bug fixes.

Description of changes: Add configuration to support spark 3.1.1(emr 6.3.0)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

larroy commented 3 years ago

NICE! Thank you.