cloudyr / aws.sqs

Amazon Simple Queue Service (SQS) API Client
https://cloud.r-project.org/package=aws.sqs
11 stars 9 forks source link

Not using iam_role on ec2 #9

Closed NelisVerhoef closed 6 years ago

NelisVerhoef commented 7 years ago

Hi, I'm trying to use the package but it does not use the ec2 role to locate the credentials.

As a work around I can just use my environmental variables as parameters like so:

queue <- create_queue(name = "...", key = Sys.getenv('AWS_ACCESS_KEY_ID'), secret = Sys.getenv('AWS_SECRET_ACCESS_KEY'), session_token = Sys.getenv('AWS_SESSION_TOKEN') )

get_queue_url("SplicedSegmentsQ", key = Sys.getenv('AWS_ACCESS_KEY_ID'), secret = Sys.getenv('AWS_SECRET_ACCESS_KEY'), session_token = Sys.getenv('AWS_SESSION_TOKEN') )

list_queues(key = Sys.getenv('AWS_ACCESS_KEY_ID'), secret = Sys.getenv('AWS_SECRET_ACCESS_KEY'), session_token = Sys.getenv('AWS_SESSION_TOKEN') )

I'm more than happy to do this, but when trying to send_message I get this error:

In sqsHTTP(query = query_args, ...) : Client error: (403) Forbidden: InvalidClientTokenId (The security token included in the request is invalid.)

The problem is that when calling .urlFromName() it does not pass my additional parameters, so either we should just pass through the additional parameters or change the definition of sqsHTTP to look like this:

sqsHTTP <- function (url = NULL, query = list(), region = Sys.getenv("AWS_DEFAULT_REGION","us-east-1"), key = Sys.getenv('AWS_ACCESS_KEY_ID'), secret = Sys.getenv('AWS_SECRET_ACCESS_KEY'), session_token = Sys.getenv('AWS_SESSION_TOKEN'), ... )

So it will default to using the environmental variables instead of setting the fields to NULL.

Or am I missing something obvious?

leeper commented 7 years ago

Can you give me the output of sessionInfo() so that I can see all your package versions?

NelisVerhoef commented 7 years ago

Sure, output from sessioninfo():

`R version 3.3.3 (2017-03-06) Platform: x86_64-redhat-linux-gnu (64-bit) Running under: Amazon Linux AMI 2017.03

locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8
[4] LC_COLLATE=en_US.UTF-8 LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C LC_ADDRESS=C
[10] LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] aws.s3_0.3.3 dplyr_0.5.0 YouiR_0.1.02 aws.sqs_0.1.10
[5] aws.ec2metadata_0.1.3 jsonlite_1.2 xml2_1.1.1 httr_1.2.1
[9] DBI_0.7 aws.signature_0.3.6

loaded via a namespace (and not attached): [1] Rcpp_0.12.10.2 digest_0.6.12 assertthat_0.1 R6_2.2.0 magrittr_1.5
[6] lazyeval_0.2.0 curl_2.3 RJSONIO_1.3-0 tools_3.3.3 base64enc_0.1-3 [11] tibble_1.2`

leeper commented 7 years ago

What does aws.signature::locate_credentials() give you? It should be picking up the environment variables and therefore correctly signing the requests without needing to explicitly pass key, etc.

You've definitely picked up on a bug in .urlFromName(). I will fix that.

NelisVerhoef commented 7 years ago

aws.signature::locate_credentials() gives me everything I need, so I get a key, secret, session_token and region. But for some reason it gives me issues.

As a work around I have made local copies of send_msg, receive_msg and delete_message, where I just changed .urlFromName. Then If I explicitly pass environmental variables everything works as I would expect.

Thanks for the quick response.

kesor commented 6 years ago

The session_token in sqsHTTP is not being read from locate_credentials or environment like it should, and default arg value is NULL which is causing temporary credentials to not work.

More details at https://github.com/cloudyr/aws.signature/issues/19#issuecomment-335428004

It does work in s3HTTP, and in your workaround above, since it is using session_token = Sys.getenv("AWS_SESSION_TOKEN") in the argument list.

leeper commented 6 years ago

Should now be resolved. Let me know if not.