403 Error when accessing S3: arcgisstore-us-east-1

[travisnburton]

Receive the following error when executing recipe[arcgis-repository::s3files2] on EC2 instance in AWS us-east-1:

Expected process to exit with [0], but received '1'
    ---- Begin output of aws s3 cp s3://arcgisstore-us-east-1/10.9.1/setups/Portal_for_ArcGIS_Linux_1091_180199.tar.gz /opt/software/archives/Portal_for_ArcGIS_Linux_1091_180199.tar.gz --region us-east-1 --no-progress ----
    STDOUT:
    STDERR: fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden
    ---- End output of aws s3 cp s3://arcgisstore-us-east-1/10.9.1/setups/Portal_for_ArcGIS_Linux_1091_180199.tar.gz /opt/software/archive

Relevant Info:

• Confirmed 403 is returned for other paths defined in this repo
• Confirmed EC2 has no issues accessing S3 (public or private buckets)

[cameronkroeker]

Hi @travisnburton,

Are you passing in aws_access_key and aws_secret_access_key or is the EC2 instance attached to an IAM Role?

https://github.com/Esri/arcgis-cookbook/blob/1c7f39341a4a967c6c73a20ee0b6bbb5b2a3dc53/templates/arcgis-enterprise-base/10.9.1/linux/arcgis-enterprise-s3files.json#L6-L12

Thanks, Cameron K.

[travisnburton]

the EC2 is attached to an IAM role, and i'm using s3files2:

{
  "arcgis": {
    "version": "10.9.1",
    "run_as_user": "arcgis",
    "repository": {
      "local_archives": "/opt/software/archives",
      "server": {
          "s3bucket": "arcgisstore-us-east-1",
          "region": "us-east-1"
      },
      "files": {
        "Portal_for_ArcGIS_Linux_1091_180199.tar.gz": {
          "subfolder": "10.9.1/setups"
        },
        "Portal_for_ArcGIS_Web_Styles_Linux_1091_180201.tar.gz": {
          "subfolder": "10.9.1/setups"
        }
      }
    }
  },
  "run_list": [
    "recipe[arcgis-repository::s3files2]"
  ]
}

[travisnburton]

Is there a specific role the EC2 instance needs to assume in order to access the bucket? Or does the bucket policy for arcgisstore-us-east-1 accept requests from all EC2 instances?

[cameronkroeker]

Thanks for clarifying the use of IAM Role rather than use of access key. I just tested both scenario's and was able to successfully download using my aws IAM Role and access key.

The setups in those buckets should be publicly available to all aws accounts so not quite sure what could be going on here. Definitely smells like a permission related issue.

As a test are you able to pass in the access key and secret to see if it yields a different result?

[travisnburton]

Thank you so much for confiming. This was indeed an IAM permission issue, the EC2 Instance Profile was not allowing access to this bucket.