search

aws / sagemaker-spark-container

The SageMaker Spark Container is a Docker image used to run data processing workloads with the Spark framework on Amazon SageMaker.
Apache License 2.0
36 stars 74 forks source link

python37-sagemaker-pyspark deleted from Dockerfile #73

Open benhyy opened 2 years ago

benhyy commented 2 years ago

I know python37-sagemaker_pyspark was removed in Apr 2021 from this commit https://github.com/aws/sagemaker-spark-container/commit/04b567630d85395d85157326f892be429a8bcfa0#diff-3cf35f33d3648c3aeaaeb5ac9cafbf8eeee4f0f7cd04bfdd00c1c4f8a209512dL62

2.4 and 3.0 containers didn't get rebuilt till Dec 2021 so problems only started popping up when I can no longer import sagemaker_pyspark

Any reason why sagemaker_pyspark was removed from the Dockerfile? Also can we add it back for future versions?