Older version of the python37-sagemaker-pyspark package in newer docker image

aws / sagemaker-spark-container

The SageMaker Spark Container is a Docker image used to run data processing workloads with the Spark framework on Amazon SageMaker.

Apache License 2.0

36 stars 74 forks source link

Older version of the python37-sagemaker-pyspark package in newer docker image #58

Open josephsolidayotto opened 3 years ago

josephsolidayotto commented 3 years ago

The current version of the python37-sagemaker-pyspark package in the (906073651304.dkr.ecr.eu-central-1.amazonaws.com/sagemaker-spark-processing:2.4-cpu) 2.4 docker image is 1.4.0. However, when I upgrade to the (906073651304.dkr.ecr.eu-central-1.amazonaws.com/sagemaker-spark-processing:3.0-cpu) 3.0 docker image, the version is 1.3.0.

Why is the version of the python37-sagemaker-pyspark package older in the newer docker image?

dp89 commented 1 year ago

HI @josephsolidayotto

Is there any package called python37-sagemaker-pyspark ? just wanted to since our data team has given package python37-sagemaker-pyspark to install ? Is it called sagemaker?