Ice ignoring credentials in ice.properties on EC2 instance

[rfein-hearst]

I'm having an issue where I set ice.s3AccessKeyId and ice.s3SecretKey in the ice.properties file, but on startup it attempts to use an IAM role anyhow.

if I pass those properties on at startup with -D, it works as expected.

| Error 2013-08-28 20:03:25,218 [localhost-startStop-1] ERROR BootStrap  - Startup failed
Message: Unable to load credentials from Amazon EC2 metadata service
    Line | Method
->>   99 | loadCredentials                  in com.amazonaws.auth.InstanceProfileCredentialsProvider
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
|     37 | getCredentials                   in     ''
|   2969 | invoke . . . . . . . . . . . . . in com.amazonaws.services.s3.AmazonS3Client
|    911 | getObject                        in     ''
|    801 | getObject . . . . . . . . . . .  in     ''
|    334 | download                         in com.netflix.ice.common.AwsUtils
|    329 | download . . . . . . . . . . . . in     ''
|    317 | downloadFileIfNotExist           in     ''
|     96 | init . . . . . . . . . . . . . . in com.netflix.ice.basic.BasicReservationService
|     89 | <init>                           in com.netflix.ice.processor.ProcessorConfig
|    179 | doCall . . . . . . . . . . . . . in BootStrap$_closure1
|    308 | evaluateEnvironmentSpecificBlock in grails.util.Environment
|    301 | executeForEnvironment . . . . .  in     ''
|    277 | executeForCurrentEnvironment     in     ''
|    334 | innerRun . . . . . . . . . . . . in java.util.concurrent.FutureTask$Sync
|    166 | run                              in java.util.concurrent.FutureTask
|   1146 | runWorker . . . . . . . . . . .  in java.util.concurrent.ThreadPoolExecutor
|    615 | run                              in java.util.concurrent.ThreadPoolExecutor$Worker
^    679 | run . . . . . . . . . . . . . .  in java.lang.Thread

Caused by FileNotFoundException: http://169.254.169.254/latest/meta-data/iam/security-credentials/
->>  106 | readResponse                     in com.amazonaws.internal.EC2MetadataClient
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
|     88 | readResource                     in     ''
|     53 | getDefaultCredentials . . . . .  in     ''
|     73 | loadCredentials                  in com.amazonaws.auth.InstanceProfileCredentialsProvider
|     37 | getCredentials . . . . . . . . . in     ''
|   2969 | invoke                           in com.amazonaws.services.s3.AmazonS3Client
|    911 | getObject . . . . . . . . . . .  in     ''
|    801 | getObject                        in     ''
|    334 | download . . . . . . . . . . . . in com.netflix.ice.common.AwsUtils
|    329 | download                         in     ''
|    317 | downloadFileIfNotExist . . . . . in     ''
|     96 | init                             in com.netflix.ice.basic.BasicReservationService
|     89 | <init> . . . . . . . . . . . . . in com.netflix.ice.processor.ProcessorConfig
|    179 | doCall                           in BootStrap$_closure1
|    308 | evaluateEnvironmentSpecificBlock in grails.util.Environment
|    301 | executeForEnvironment            in     ''
|    277 | executeForCurrentEnvironment . . in     ''
|    334 | innerRun                         in java.util.concurrent.FutureTask$Sync
|    166 | run . . . . . . . . . . . . . .  in java.util.concurrent.FutureTask
|   1146 | runWorker                        in java.util.concurrent.ThreadPoolExecutor
|    615 | run . . . . . . . . . . . . . .  in java.util.concurrent.ThreadPoolExecutor$Worker
^    679 | run                              in java.lang.Thread

[fangji]

@rfein-h As specified in readme, ice.s3AccessKeyId and ice.s3SecretKey need to be set as runtime parameters. They should not be defined in ice.properties file.

Thanks -Fang

[nmcserra]

@fangji Is there a reason why they can't be defined in ice.properties? That would be a neat way to hide them if you're not using AWS instance metadata. If you pass them as runtime java parameters any user able to list the running processes will be able to see the credentials, it's usually not best practice to pass sensitive data in command line options precisely because of this.

Thanks, Nuno

[fangji]

@nmcserra The idea is that you don't need to specify aws credentials while running Ice in the cloud. Providing the option to specify credentials as runtime parameters is only to enable people debugging/running ice on their local desktop.

[pjjw]

some of us run ice outside of aws, oddly enough. this is problematic because it means that your keys will be visible to anyone who can see the task's command line (i.e. anyone on the box)

[nyetsche]

I just created a quick and dirty fix here - https://github.com/nyetsche/ice/commit/e6eb58f971a3f9d3dcc882abc0ed3190498acc89

I didn't add a pull request because I just ignored the System.getProperty() option so you can only use the ice.properties file. If there's interest I can rewrite and do a pull request.

Otherwise, there's a quick solution if you want to keep your credentials in that file.

[jantman]

@fangji The documentation (https://github.com/Netflix/ice#basic-setup) says:

1.4 If running locally, set the following system properties at runtime. ice.s3AccessToken is optional.

ice.s3AccessKeyId=<accessKeyId>
ice.s3SecretKey=<secretKey>
ice.s3AccessToken=<accessToken>

I think it's a valid assumption that these will work if not running "locally" as well... at the least, this could be made more clear in the docs.