Allow functions to temporarily unset values found in ~/.aws/credentials?

[cboettig]

Package functions allow us to override the default credentials found in ~/.aws/credentials with alternative values, but we have no comparable way to merely "blank" or unset those credentials. This creates unexpected behavior and errors that can be difficult to debug. For example, if you have no aws_secret_access_key defined in ~/.aws files, then the following code successfully uploads to this public upload-only AWS bucket as expected:

library("aws.signature")
library("aws.s3")
write.csv(iris, 'iris.csv')
aws.s3::put_object('iris.csv',
                   bucket = "submissions",
                   region = "data",
                   base_url = "ecoforecast.org")

If, however, you have your own secret access key in an ~/.aws directory, the same code will throw a 403 Access Forbidden error, even though the bucket does not require access, because it tries to authenticate anyway with your secret key. Note that attempting to set "AWS_SECRET_ACCESS_KEY" = "" env var to an empty string or passing secret="" to put_object does not help the situation. Here's a fully reproducible minimal example of the error; but note it will overwrite any existing ~/.aws/credentials

library("aws.signature")
library("aws.s3")
write.csv(iris, 'iris.csv')
dir.create("~/.aws")
writeLines(
"[default]
aws_access_key_id = ACCESS_KEY
aws_secret_access_key = SECRET_KEY
aws_session_token = TOKEN",
"~/.aws/credentials")

aws.s3::put_object('iris.csv',
                   bucket = "submissions",
                   region = "data",
                   secret = "", 
                   base_url = "ecoforecast.org")

Would you consider a way to temporarily unset default credentials, such as allowing a blank string or NULL value?

This error is also difficult for users to diagnose and debug. Being able to 'blank' credentials out would probably be useful in other contexts as well. (e.g. I'm not a security expert but in general I could imagine a user wanting finer-grained control of whether or not package functions were touching secure credentials when they weren't needed).

[grimbough]

Pretty old issue, but I agree with @cboettig, there are occasions where it'd be really nice to explicitly choose to ignore stored credentials. For example, consider the following example where I'm accessing a file in a public bucket that doesn't require any credentials. If credentials are found anywhere by aws.signature I can't figure out any way to access the bucket e.g.

library(aws.s3)
aws.s3::object_exists(object = "zarr/v0.4/idr0076A/10501752.zarr/0/.zarray", 
                      bucket = "idr", 
                      region = "",
                      base_url = "uk1s3.embassy.ebi.ac.uk")
#> [1] TRUE

## Set a key and secret pair
Sys.setenv(
  "AWS_ACCESS_KEY_ID" = "NOT_A_REAL_KEY",
  "AWS_SECRET_ACCESS_KEY" = "ALSO_NOT_REAL"
)

## Now we can no longer access the public bucket
aws.s3::object_exists(object = "zarr/v0.4/idr0076A/10501752.zarr/0/.zarray", 
                      bucket = "idr", 
                      region = "",
                      base_url = "uk1s3.embassy.ebi.ac.uk")
#> Client error: (403) Forbidden
#> [1] FALSE

Passing key = "" and secret = "" to the function doesn't help, because aws.signature:::check_for_user_supplied_values() actively discounts the empty string as input and will continue looking in other locations.

aws.s3::object_exists(object = "zarr/v0.4/idr0076A/10501752.zarr/0/.zarray", 
                      bucket = "idr", 
                      region = "",
                      base_url = "uk1s3.embassy.ebi.ac.uk", 
                      key = "",
                      secret = "")
#> Client error: (403) Forbidden
#> [1] FALSE

It'd be great to have some mechanism to still access public buckets even if I have credentials for somewhere else stored on my system.