Unable to use Amazon S3 endpoints

[jpds]

After #45, I decided to try some HTTP based S3 endpoints, taking inspiration from https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html

endpoint: https://company-drone-cache.s3-eu-west-2.amazonaws.com/

Oddly results in:

msg="Uploading to bucket organization at project/branch/archive.tar"

Not sure why one would default to organization for the bucket name. Maybe just make bucket name a configuration option? Also I see nothing [created] in my S3 account by this name either.

Take 2:

endpoint: https://s3-eu-west-2.amazonaws.com/company-drone-cache

msg="Endpoint: s3-eu-west-2.amazonaws.com/company-drone-cache does not follow ip address or domain name standards."

[tboerger]

Just don't define any endpoint for aws s3, it's confirmed that it works without issues.

If you want to customize the path just use the documented attributes.

[benwilson512]

How do you define which bucket to use? The docs make no mention of "bucket".

[tboerger]

The bucketname gets defined by the first segment of the path, which defaults to the repo owner. You can overwrite it if you define a root attribute.

[benwilson512]

I'm happy to PR additional docs, but I'm a bit unclear on where the file containing the drone docs sits. I do feel that an S3 specific example would be an important documentation contribution.

[tboerger]

The documentation gets built from https://github.com/drone/drone-plugin-index/blob/master/content/drone-plugins/drone-s3-cache/index.md, on http://plugins.drone.io/drone-plugins/drone-s3-cache/ there is also a link at the bottom: Is there a mistake on this page? Please let us know or edit this page.

[tboerger]

And generally the docs for this plugin are hosted on http://plugins.drone.io/drone-plugins/drone-s3-cache/

[Iced-Sun]

What about AWS China region?

- name: rebuild
  image: plugins/s3-cache
  settings:
    pull: true
    endpoint: https://s3.cn-northwest-1.amazonaws.com.cn
    region: cn-northwest-1
#    root: /AAA
    access_key: KEY
    secret_key: KEYKEY
    rebuild: true
    debug: true

With the above config, drone just hangs forever, with the last message of:

time="2019-01-10T10:01:30Z" level=info msg="Uploading to bucket AAA at BBB/develop/archive.tar"

[Pajk]

@sunbing81 FYI to use a non-default S3 region you have to set also the region to make it work. You can find it in the AWS docs.

[Iced-Sun]

@sunbing81 FYI to use a non-default S3 region you have to set also the region to make it work. You can find it in the AWS docs.

Thanks for the reply.

My bad. I forgot to uncomment the 'endpoint' & 'region' config lines when pasting. To clarify, with the endpoint and region configs, s3-cache hangs forever.

The comment is edited (https://github.com/drone-plugins/drone-s3-cache/issues/46#issuecomment-453041779).

[tboerger]

I'm personally not using s3 at all, only services like Minio.

You could try the plugin manually as described in the readme to figure out how it works for these endpoints.

[masterkain]

I'm trying to make this work with scaleway's s3-compatible object storage, but with this config I get:

    access_key: xxxx
    secret_key: xxxx
    server: https://s3.fr-par.scw.cloud
    region: fr-par
    root: myexistingbucket

level=warning msg="Cache could not be restored The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'fr-par'"

any idea why there's still a wrong region mentioned?

[tboerger]

Please create a separate issue with detailed information. This is simply a different thing.