ipfs / infra

Tools and systems for the IPFS community
MIT License
133 stars 41 forks source link

I need an S3 Bucket :) #462

Closed mikeal closed 5 years ago

mikeal commented 5 years ago

As part of the metrics work I need some storage.

All I need is a single S3 bucket that is the world readable and some credentials to write into it.

It really should be S3 because I'm also using some functions built with Zeit that end up getting deployed on Lambda, so it will be a lot faster if the whole thing is in S3.

eefahy commented 5 years ago

All I need is a single S3 bucket that is the world readable

easy peasy

and some credentials to write into it.

more info needed. I'd love to not mint a long term IAM user for access to this bucket so it would be good to know if there's a way to provision an AWS resource with a role that can mint short(ish) term tokens for access to the bucket. By "deployed on Lambda" do you mean AWS Lambda? If so, how does Zeit handle an IAM role for Lambda? That would be the place to grant write access to the S3 bucket

mikeal commented 5 years ago

If so, how does Zeit handle an IAM role for Lambda?

Zeit is provisioning in its own account and proxy billing me for it. Right now the load is light enough that this is just on my personal account.

We don't need to have an IAM role that Zeit knows about. I just need a token that I can use in the Lambda function which will access S3 like any other client would.

I just need something I can plug into the JS SDK, either an accessKey/secretKey or a credentials file. https://www.npmjs.com/package/aws-config

eefahy commented 5 years ago

I've deployed node lambdas before and they require an execution role that has the privileges attached to it. All AWS SDKs know how to farm out to that role to mint short term credentials. Perhaps you can point me to the Zeit docs so I can see how they handle this?

mikeal commented 5 years ago

Not sure, docs are here https://zeit.co/docs/

On Mon, Dec 10, 2018, 7:12 PM eefahy <notifications@github.com wrote:

I've deployed node lambdas before and they require an execution role https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/loading-node-credentials-lambda.html that has the privileges attached to it. All AWS SDKs know how to farm out to that role to mint short term credentials. Perhaps you can point me to the Zeit docs so I can see how they handle this?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ipfs/infra/issues/462#issuecomment-446057056, or mute the thread https://github.com/notifications/unsubscribe-auth/AAACQ3_E6qj8OL3HbrUptJXFiinniiN6ks5u3yKXgaJpZM4ZMbyL .

eefahy commented 5 years ago

Bummer. They are obvs very engineered for stateless apps so I guess a user account with a minted key is what it needs to be. How long do you need this set up for? Is this a long or short term thing?

Also, do you have a project name for this work? What should I call the bucket and/or username for access to it? Do you have a preferred region for the bucket?

mikeal commented 5 years ago

The metrics project is long term. We may find a better way to process this data in the future and be able to deprecate this solution but that's at least a year out.

eefahy commented 5 years ago

Also, do you have a project name for this work? What should I call the bucket and/or username for access to it? Do you have a preferred region for the bucket?

thoughts?

mikeal commented 5 years ago

Sorry, didn't see that, ipfs-metrics :)

eefahy commented 5 years ago

I made the bucket and the user. Do you have access to any 1pass vaults?

mikeal commented 5 years ago

Following up in private messages :)

eefahy commented 5 years ago

Happy doing business with you!

Closed via https://github.com/protocol/ad-hoc-infra/pull/1