search

aws / sagemaker-spark-container

The SageMaker Spark Container is a Docker image used to run data processing workloads with the Spark framework on Amazon SageMaker.
Apache License 2.0
36 stars 74 forks source link

s3-us-east-1.amazonaws.com’s server IP address could not be found. #99

Open pancodia opened 2 years ago

pancodia commented 2 years ago

While building the docker image, it needs to configure a emr yum repo and install packages from this yum repo.

When I set REGION to be ue-east-1, docker build cannot resolve the yum repo.

Then I found this is caused by the following line.

https://github.com/aws/sagemaker-spark-container/blob/3ccb7298f1cce587fbbdbb9f5ff05962cfc7a31d/spark/processing/3.1/py3/yum/emr-apps.repo#L3

Specifically, when REGION=us-east-1, this url cannot be resolved. This is confirmed when I tried to visit this url in my browser, I got "s3-us-east-1.amazonaws.com’s server IP address could not be found."

From the s3 endpoint document, it seems that the format of the url should be https://s3.REGION.amazonaws.com/ .... Is my understanding right?