IOT Security Credentials from EC2 Instance Metadata

[LeviDooley]

So, I'm running a service in an EC2 instance which publishes to iot using this device sdk. I have decided to use the websocket protocol for this. The websocket protocol requires a Access Key ID and a Secret Key, but I would rather specify a role in my EC2 instance profile.

According to many places in the docs, "the AWS SDKs... automatically get the credentials from the EC2 instance metadata service and use them."

So, as long as a role is assigned to the EC2 instance, the sdk should grab the keys automatically. I don't see that behavior from this code though. I get the following error.

{ Error: ENOENT: no such file or directory, open 'C:\Users\Administrator\.aws\credentials'
    at Error (native)
    at Object.fs.openSync (fs.js:641:18)
    at Object.fs.readFileSync (fs.js:509:33)
    at DeviceMQTT.DeviceClient (C:\Users\Administrator\Services\iot-service\node_modules\aws-iot-device-sdk\device\index.js:446:43)
    at DeviceMQTT (C:\Users\Administrator\Services\iot-service\server.js:27:9)
    at Request.awsIot.device.request [as _callback] (C:\Users\Administrator\Services\iot-service\server.js:86:20)
    at Request.self.callback (C:\Users\Administrator\Services\iot-service\node_modules\request\request.js:188:22)
    at emitTwo (events.js:106:13)
    at Request.emit (events.js:191:7)
    at Request.<anonymous> (C:\Users\Administrator\Services\iot-service\node_modules\request\request.js:1171:10)
  errno: -4058,
  code: 'ENOENT',
  syscall: 'open',
  path: 'C:\\Users\\Administrator\\.aws\\credentials' }
Failed to read credentials from C:\Users\Administrator\.aws\credentials
To connect via WebSocket/SigV4, AWS Access Key ID and AWS Secret Key must be passed either in options or as environment variables; see README.md

I would just provide the keys explicitly, but AWS swaps them out periodically.

const awsIot = require("aws-iot-device-sdk");
const aws = require("aws-sdk");

var aws_params = {
    region: "us-west-2"
};
aws.config.update(aws_params);

var wsClient = new awsIot.device({
    region: aws.config.region,
    protocol: 'wss'
    port: 443,
    host: 'xxxxxxxxxxx.iot.us-west-2.amazonaws.com'
});

[groteworld]

Hey @LeviDooley,

I'd suggest using the JavaScript aws-sdk.IotData API for this. This device SDK is more closely tied to the X.509 cert authentication, but I don't know everything, so hopefully I find out something new for this too!

[LeviDooley]

@groteworld Thanks for that direction. I was not aware that such a thing existed. 😄

It might work for now, since I'm only publishing to the IoT topic right now, but I find it a bit concerning that it only has four methods, deleteThingShadow, getThingShadow, publish, and updateThingShadow. It's very odd that it has no ability to subscribe to a topic. Seems like it's made for applications that don't have the ability to stay connected to a websocket/mqtt connection.

However, I will need the ability to subscribe to topics in the future, so it's not a permanent solution.

[fengsongAWS]

Hi @LeviDooley , The AWS IoT Node.js Device SDK is different than AWS Node.js SDK. So, you are right that IoT SDK cannot automatically grab your ec2 role credentials. Right now, there are three ways of using websocket using aws credentials. This is same either you are using ec2 host or the others.

1. You can pass credentials when you instantiate device class.
2. You can explicitly export your aws credentials as environment variables.
3. You can have a default aws credential profile located at ~/.aws

[LeviDooley]

@fengsongAWS Thank you. I was beginning to suspect that. Do you have any recommendations for seamlessly switching between the temporary AWS credentials provided by the EC2 instance metadata?

Regarding EC2 instance metadata, "These security credentials are temporary and we rotate them automatically. We make new credentials available at least five minutes prior to the expiration of the old credentials."

Is there a way to update an existing awsIot.device to start using new credentials? Or does the device have to be torn down, and re-initialized?

For example, if I do it this way, it works up until the keys expire.

var wsClient;

request({
    method: 'GET',
    url: `http://169.254.169.254/latest/meta-data/iam/security-credentials/roleName`
}, (err, resp) => {
    var keys = JSON.parse(resp.body);
    console.log(`
        AccessKey: ${keys.AccessKeyId}, 
        SecretKey: ${keys.SecretAccessKey}, 
        SessionToken: ${keys.Token},
        Expiration: ${keys.Expiration}
    `);

    wsClient = new awsIot.device({
        region: aws.config.region,
        protocol: 'wss',
        accessKeyId: keys.AccessKeyId,
        secretKey: keys.SecretAccessKey,
        sessionToken: keys.Token,
        port: 443,
        host: "xxxxxxxxxxx.iot.us-west-2.amazonaws.com"
    });
});

So, my question is, when it expires, do I have to re-initialize by creating a new device to replace the old one? Like so...

request({
    method: 'GET',
    url: `http://169.254.169.254/latest/meta-data/iam/security-credentials/roleName`
}, (err, resp2) => {
    var keys2 = JSON.parse(resp2.body);

    wsClient = new awsIot.device({
        region: aws.config.region,
        protocol: 'wss',
        accessKeyId: keys2.AccessKeyId,
        secretKey: keys2.SecretAccessKey,
        sessionToken: keys2.Token,
        port: 443,
        host: "xxxxxxxxxxx.iot.us-west-2.amazonaws.com"
    });
});

Or is there some way to update the credentials of the existing wsClient seamlessly? Something along the lines of this...

request({
    method: 'GET',
    url: `http://169.254.169.254/latest/meta-data/iam/security-credentials/roleName`
}, (err, resp2) => {
    var keys2 = JSON.parse(resp2.body);

    wsClient.updateCreds({
        accessKeyId: keys2.AccessKeyId,
        secretKey: keys2.SecretAccessKey,
        sessionToken: keys2.Token
    });
});

[fengsongAWS]

You can try updateWebsocketCredentials like https://github.com/aws/aws-iot-device-sdk-js/blob/master/device/index.js#L830.

Another way is using AWS Cognito service which provides you a temporary aws credentials. More information could be found here. https://aws.amazon.com/cognito/

[LeviDooley]

@fengsongAWS Cool. 😄 Thank you. I don't know how I missed that function. That solves my issue.

I'm not sure how cognito would help me though. From what I understand, cognito is for verifying user identities, but this isn't a user, it's an ec2 instance.