search

aws / sagemaker-spark-container

The SageMaker Spark Container is a Docker image used to run data processing workloads with the Spark framework on Amazon SageMaker.
Apache License 2.0
36 stars 74 forks source link

clean up python 3.9 installation #87

Closed guoqiao1992 closed 2 years ago

guoqiao1992 commented 2 years ago

Issue #, if available: After install python 3.9, image size increased 400MB. Clean up container after installing python 3.9

Description of changes:

  1. Instead of installing smspark in Pipfile, install it using pip
  2. Change the dependency between init and build-container-library
  3. Remove python3.7 related dependencies

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.