meltwater / drone-cache

A Drone plugin for caching current workspace files between builds to reduce your build times
https://underthehood.meltwater.com/blog/2019/04/10/making-drone-builds-10-times-faster/
Apache License 2.0
338 stars 81 forks source link

Unable to read from S3 buckets with TLS/SSL required #211

Open bbs-csel opened 2 years ago

bbs-csel commented 2 years ago

Describe the bug We've recently enforced TLS/SSL on all traffic to and from our S3 buckets including the one holding the cached artifacts from our Drone pipelines. After we've forced this, the plugin is unable to read and write from/to the bucket:

[IMPORTANT] restore cache, restore failed, 2 errors: download from
<[redacted]/7b28d59f43753746111f159b0d67bea8/.sbt> to <.sbt>, extract files from downloaded archive, pipe reader failed, get file from storage backend, pipe writer failed, get the object, AccessDenied: Access Denied\n\tstatus code: 403, request id: [redacted], host id: [redacted];
download from <[redacted]/7b28d59f43753746111f159b0d67bea8/.ivy2> to <.ivy2>, extract files from downloaded archive, pipe reader failed, get file from storage backend, pipe writer failed, get the object, AccessDenied: Access Denied\n\tstatus code: 403, request id: [redacted], host id: [redacted];

We've attempted, on the same node (EC2) with the same IAM role to read/write from the bucket using e.g. the AWS CLI and that works without issue. Same goes for e.g. Java applications using the AWS SDK for Java, so it seems related to this application and/or the Go SDK (but haven't been able to find any chatter around that SDK being broken).

The policy used on s3 is the one below, as described by AWS here: https://aws.amazon.com/premiumsupport/knowledge-center/s3-bucket-policy-for-config-rule/.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowSSLRequestsOnly",
            "Effect": "Deny",
            "Principal": "*",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::[bucket]/*",
                "arn:aws:s3:::[bucket]"
            ],
            "Condition": {
                "Bool": {
                    "aws:SecureTransport": "false"
                }
            }
        }
    ]
}

To Reproduce

  1. Create and S3 bucket with above policy
  2. Try to either restore or build a cache in the bucket
  3. See error in the plugin logs

Expected behavior The plugin reading and writing to the bucket unimpacted by the change.

bbs-csel commented 2 years ago

It seems like this might be related to the same issue described in https://github.com/meltwater/drone-cache/issues/144 which also seems to be around issues with lack of TLS by default.