Question: how to set credentials and region?

[AndreasPizsa]

Hi Izaak @izaakschroeder, thank you for this wonderful idea and implementation!

May I ask you for your advice, as I'm having trouble setting AWS options and am not sure where and how to set them correctly. I understand that vinyl-s3 builds upon aws-sdk, so I read their docs and had some success.

I tried setting AWS options with AWS.config.update before instantiating vinyl-s3, but unfortunately that had no effect at all (and I don't quite understand why). What did work was setting environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY - fair enough.

Where I'm still having difficulty is the region.

I tried setting AWS_DEFAULT_REGION but unfortunately that was not successful. I then tried AWS.config.update({region:'eu-west-1'});, but I'm still getting

PermanentRedirect: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.

I'm a bit stuck here and am hoping you might have an idea?

Thank you very much for your help!

Best, Andreas

UPDATE: setting AWS_REGION worked - but I'm still wondering if and how it's possible to set options programmatically? Thank you!

[izaakschroeder]

Hey @AndreasPizsa!

So the reason the above error is happening is that in the vinyl-s3 module I list aws-sdk as a direct dependency (so you don't have to install anything); the problem is that it means vinyl-s3 has its OWN copy of the AWS variable; the one you're trying to access is different, even though it looks the same.

To address this I could list aws-sdk in peerDependencies instead, but that would also mean that you must then install aws-sdk as part of your package's dependencies in addition to having vinyl-s3.

You will also note that the src and dest methods can take the S3 object as an options parameter. So you can s3.src(... { s3: new AWS.S3({ accessKeyId: ... }) }) in your options. See here for reference.

I'm not certain of this, but I think you might also be able to pass the parameters you want as part of the src or dest functions. e.g. s3.src('s3://foo/bar/*.jpg?region=eu-west-1'). I'm not sure exactly on the syntax or variables for this.

I recommend keeping them in environment variables since it's supported by the AWS SDK out of the box and allows you to transition easily to other safer authentication methods where supported (e.g. on an EC2 instance you can use the role assigned to the instance instead).

[izaakschroeder]

Closing due to lack of activity.

[jamestalmage]

What did work was setting environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY

.

You will also note that the src and dest methods can take the S3 object as an options parameter. So you can s3.src(... { s3: new AWS.S3({ accessKeyId: ... }) }) in your options.

.

I recommend keeping them in environment variables since it's supported by the AWS SDK out of the box and allows you to transition easily to other safer authentication methods where supported

This all should get documented in the README.