search

nienbo / cache-buildkite-plugin

Tarball, Rsync & S3 Cache Kit for Buildkite. Supports Linux, macOS and Windows
https://buildkite.com/plugins
MIT License
67 stars 39 forks source link

docs(readme): Improve Documentation On S3 Usage #3

Closed bradleyess closed 4 years ago

bradleyess commented 4 years ago

Howdy 🤠

Very excited to dig into using this plugin, I believe it solves some very fundamental challenges w/ Buildkite especially as projects grow!

We are hoping to cache our node_modules directory between builds and shipping them up to S3. I've had a poke around to try and get this working but believe the documentation is lacking some solid real life examples.

Question(s)

  1. What is an S3 profile? Are we talking IAM Role here?
  2. How do we use the saved cache as opposed to saving the cache? I believe this is possible with the plugin but again, it's not clear how this is achieved.
gencer commented 4 years ago

Hey @bradleyess!

Nice to see this plugin works well on your case! Let me answer your questions:

  1. Yes. As you see in hooks/, s3 cache uses AWS CLI. AWS CLI has a configuration file in your local user path. This configuration file can have multiple profiles. For example; You can 2 account/profiles one for generic s3 account and second for cache purposes with very limited access. Let me show you with example:
[default]  <-- this is "default" profile. No need to specify profile for this.
aws_access_key_id=AKIAIOSFODNN7EXAMPLE
aws_secret_access_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

[user1]  <-- this is "user1" profile
aws_access_key_id=AKIAI44QH8DHBEXAMPLE
aws_secret_access_key=je7MtGbClwBF/2Zp9Utk/h3yCo8nvbEXAMPLEKEY

As you can see we have 2 different profiles to work with AWS CLI. When you specify this profile with cache plugin, we will use the specified profile's credentials instead the default one. To be more explanatory, you can have more than one cache endpoint with this way.

See more details in here: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html Also see profile argument on cli here: https://github.com/gencer/cache-buildkite-plugin/blob/master/hooks/post-command#L19

  1. I'm bit lost in this question actually, What do you mean by saying "using saved cache opposed to saving cache"?
gencer commented 4 years ago

I'm closing this issue as no response given in a manner time.

Feel free to re-open if you have something to add.