Lambda Function to Connect to AWS IoT and Publish MQTT Messages

[nbibin001]

Hi I was trying to develop a lambda function which will connect to the AWS thing and publish an MQTT message, I want to receive these published messages on a raspberry Pi which is connected to the thing as well, and subscribed to the same topic. I managed to create the thing, certificate and connect to it. But the lambda function seems to not send the MQTT message. I am attaching my sample code here in the index.js. I kept all my certificates and keys in a folder and zipped and uploaded. Since I am testing this lambda function now, I am not using the event to trigger the connection and message publishing, I just did it like whenever the function is called , it will connect and publish the message. The function seems to be running without error, but I dont see any message coming to my Pi when this runs, (I have a python code running at the Pi side which is subscribed to the same topics, and it is connected to AWS as well). Below is the lambda code I am using, could anyone please help me if I am missing something here ?

var awsIot = require('aws-iot-device-sdk');
var deviceName = "Bibin_Pi_3";  // Updated by Bibin from AWS IOT Thing name

var thingShadows = awsIot.thingShadow({
keyPath: './private.pem.key',
certPath: './certificate.pem.crt',
caPath: './rootCA.pem',
clientId: deviceName,
region: "eu-east-1",
});

var ctx = null;

exports.handler = function (event, context) {
    ctx = context;

    thingShadows.on('connect', function() {
    thingShadows.register(deviceName);
    console.log("Connected...");
        console.log("Registering...");
});

thingShadows.on('message', function(topic, payload) {
        console.log('message', topic, payload.toString());        
});

    thingShadows.publish('qrio/test/pi/voice', 'BedroomLightOn', function(){
    var cardTitle = "Bedroom Lamp on";
    var repromptText = "";
    var sessionAttributes = {};
    var shouldEndSession = false;
    var speechOutput = "Your bedroom light is on. ";
    repromptText = "Your bedroom light is on. ";
    var shouldEndSession = false;   
    });
    console.log('value1 =', 'Bibin');
    ctx.succeed();
};

[fengsongAWS]

HI @nbibin001 ,

Thanks for using AWS IoT device SDK.

I would suggest try to use AWS IoT console to subscribe the topics and see if you receive the messages. This is to make sure that your lambda function published messages correctly. And then, you could also try to publish messages to the topic using same client and see if your raspberry pi could receive the topics. This will help you find out which part causes the problem.

Thanks, Fengyi

[fengsongAWS]

Just to follow up with the comments, we would suggest to using AWS SDK in Lambda function because AWS IoT Device SDK uses MQTT connection which is long lived connection while AWS SDK provides control plane APIs in HTTP. Since Lambda function is serverless and not designed for long live connection, AWS SDk would be more suitable to be used in Lambda function.

Fengyi

[nbibin001]

Hi Fengyi, thanks a lot for your suggestion. It started to work for me with the AWS IOT Device SDK. I guess the problem was in the above code snippet above I used the deviceName as the Thing name string. I guess it should be the Thing ID. When I made this change it started to work and the mqtt messages I was able to receive on my raspberry pi connected to the thing. Regarding the AWS SDK, if I use that can I still use mqtt messaging ? Also where do I get the AWS SDK. May be i can google it, just checking if there is a known working version. Appreciate your help.

[fengsongAWS]

AWS SDK uses HTTP which does not support long live connection so you cannot use mqtt messaging by using AWS SDK but you could publish messages by using HTTP control plane API. More details can be found here. https://aws.amazon.com/tools/

[nbibin001]

Hi Fengyi, thanks for the information. I guess I would like to keep the mqtt messaging at my Raspberry Pi side, so may be I will continue to use AWS IoT device SDK. One other question though is as I mentioned earlier, now I got the lambda function publish an MQTT message and at the Raspberry Pi side, I am using a python code to connect to the Thing using the same certificates and keys, and subscribing to the message topics on which Lambda function is publishing. The python code receives the published messages everything works well. Now i wanted to replace the python code with Node js . So I just wrote a simple code based on the given examples, to subscribe to the message topic published by lambda function, and display the message when the MQTT message arrives. So it shows me that it is connected to the Thing, but never receives the message. If I run the python script and Node js script together in my Pi, I can see that the Python is still receiving the MQTT messages , but not the Node js. I am missing something, cant figure out yet. Could you please help. Really appreciate your help.

var awsIot = require('aws-iot-device-sdk');

var myThingName = 'xxxxxxxxxxx.iot.us-east-1.amazonaws.com';

var device = awsIot.device({
    keyPath: './private.pem.key',
    certPath: './certificate.pem.crt',
    caPath: './rootCA.pem',
    clientId: myThingName,
    region: 'us-east-1'
});

device
  .on('connect', function() {
    console.log('connect');
    device.subscribe('qrio/test/pi/voice', function(error, result) {
        //console.log(result);
  });
});

device
  .on('message', function(topic, payload) {
    console.log('message', topic, payload.toString());
  });

[fengsongAWS]

Hi @nbibin001 ,

I am not able to reproduce your problem. The code above works for me. Are you able to run the example of device-example.js ? There's a few things you can check. The 'xxxxxxxxxxx.iot.us-east-1.amazonaws.com' is supposed to be your endpoint and it is not necessarily to be your clientId. The thing name is supposed to be same as the name of the thing you created in the IoT console. It is usually needed when you register your thing shadow. The client ID could be any arbitrary string. In addition, two connections cannot share the same clientId. So, please make sure you are not using the same clientId at the same time for your clients (e.g. python client and node.js client).

Also, please feel free to paste your working python code, which could help us resolve the problem.

Thanks, Fengyi

[nbibin001]

Hi Fengyi, thanks a lot for the suggestions, I changed the myThingName variable in the above code with my actual Thing name which is 'Bibin_Pi_3', and now it worked :+1: It was little confusing in a way that in the lambda node js function I had to use the myThingName to be 'xxxxxxxxxxx.iot.us-east-1.amazonaws.com' to make it connect and publish the message. But at my Raspberry Pi side, I had to use the thing name 'Bibin_Pi_3', after your suggestion. It will be great if you can just explain why this is , I guess that will help me understand this better.

[fengsongAWS]

The usage of client ID you used in Lambda shouldn't be different than running sdk locally. You could still use any client ID you want in your lambda function.

The reason we encourage you to use AWS SDK in lambda function is that lambda function is serverless short-lived function. You could still receive MQTT messages from Broker(AWS IoT) by useing AWS SDK. The AWS IoT Gateway will translate the HTTP publish request to MQTT message for you, so that any MQTT client subscribing to that topic could still receive your published messages.

Thanks, Fengyi

[fengsongAWS]

Close this issue for now and feel free to reopen it or post other questions you may have.

[knownothing8]

I am learning lambda function. Do you know how to upload certificates file for lambda function to use?

[fengsongAWS]

hi @knownothing8 , You can zip the your node package and upload to lambda directly. However, using AWS IoT Device SDK is not recommended because lambda does not maintain long live connection so traditional pub/sub model does not work out in this case. You can use AWS SDK https://github.com/aws/aws-sdk-js which is a HTTP based model.

[knownothing8]

Hi FengSong, from your old comment, http based model can only support publish, but cannot subscribe. I have physical IOT device which need both pub/sub. As such, I need mqtt from AWS IOT device SDK.

1. If you think it is not recommended to use AWS IOT Device SDK for mqtt, perhaps should I find other MQTT broker such as https://www.pubnub.com/

2. Another issue. This is zip file hierarchy. However, lambda function complained about invalid keypath Error:

   "errorMessage": "Exception: Error: Invalid \"keyPath\" option supplied."

index.js file

var awsIot = require('aws-iot-device-sdk');

var mqtt_config = { "certPath": './certs/certificate.pem.crt', "caPath": './certs/rootCA.key', "keyPath": './certs/private.pem.key', "port": 8883, "host": host, "clientId": deviceName, "region":"us-east-1", "debug":true, };

client = awsIot.thingShadow(mqtt_config);

├── certs │ ├── certificate.pem.crt │ ├── private.pem.key │ └── root-CA.crt ├── index.js ├── node_modules │ └── aws-iot-device-sdk

[fengsongAWS]

1. The reason not recommended is because of the nature of lambda. Lambda function does not support long live connection. AWS IoT Device SDK is designed for mqtt pub/sub model with AWS IoT Cloud service.

2. Please double check your path and naming. For example, I see different naming for root-CA.crt vs rootCA.key For more information, please refer to http://docs.aws.amazon.com/lambda/latest/dg/nodejs-create-deployment-pkg.html

[knownothing8]

@fengsongAWS Thanks for the reply. I finally resolve both issues.

You are right about the recommendation. Since I only need Alexa skill to send a command to AWS iot through lambda function, only MQTT publish is needed. I assume MQTT subscribe is not recommended inside Lambda function since Lambda function does not support long live connection while MQTT subscribe require long live connection. Here are several issues which I resolves. Hopefully, it will be useful to anyone encountering same issue.

1. Key files link

After zip those files before upload, the zip has first foldername hierarchy. in lambda function, look like it consider foldername as root directory. As such, foldername need to be included in file name link For example, folder name is index. link should be "./index/certs/" instead of "./certs/" just like following

var mqtt_config = { "keyPath": "./index/certs/xxx-private.pem.key", "certPath": "./index/certs/xxx-certificate.pem.crt", "caPath": "./index/certs/root-CA.key", "port": 8883, "host": endName, "clientId": deviceName, "region":"us-east-1", "debug":true, };

foldername ├── certs │ ├── certificate.pem.crt │ ├── private.pem.key │ └── root-CA.crt ├── index.js ├── node_modules │ └── aws-iot-device-sdk

2. Test lambda function, but did not see "Connected to AWS IoT" in the log output. It means MQTT connection failed. After I fixed policy resource to be "*", MQTT connection passed. Here is the explanation.

   thingShadows..on("connect",function(){
       console.log("Connected to AWS IoT");
   });

aws iot thing policy is not set correctly for the certificate. Here is the step to review the policy

• go to aws console and click aws iot https://console.aws.amazon.com

• check the policy(which is used for the certificate) and make sure that action and resource are "iot:" and ""

  "Effect": "Allow", "Action": "iot:", "Resource": ""

3. ClientID for MQTT is just an ID. can be any name. It is not the thing name registered in AWS IOT.

4. Topic does not need to be registered. you can create any topic name in your lambda function. Topic name is not thing name or shadow name registered in ASW IOT.

[bxem44]

How would a lambda function which only publishes messages look like? Any sample code?

[soundarsurya]

@fengsongAWS while i execute my lambda function i got this error in my console log

{ "errorMessage": "Invalid connect options supplied.", "errorType": "Error", "stackTrace": [ "Object.Module._extensions..js (module.js:579:10)", "Module.load (module.js:487:32)", "tryModuleLoad (module.js:446:12)", "Function.Module._load (module.js:438:3)" ] }

how could it be fixed...??? i have almost complete help me to finish it

thanks in advance

[soundarsurya]

hi @nbibin001, well i have used the above method you said but while using thingshadow how it could be connect?

{ "errorMessage": "Invalid connect options supplied.", "errorType": "Error", "stackTrace": [ "new ThingShadowsClient (/var/task/node_modules/aws-iot-device-sdk/thing/index.js:123:30)", "Object.ThingShadowsClient [as thingShadow] (/var/task/node_modules/aws-iot-device-sdk/thing/index.js:89:14)", "Object. (/var/task/index.js:6:26)", "Module._compile (module.js:570:32)", "Object.Module._extensions..js (module.js:579:10)", "Module.load (module.js:487:32)", "tryModuleLoad (module.js:446:12)", "Function.Module._load (module.js:438:3)" ] }

i'm getting this error

[soundarsurya]

@knownothing8 how pubnub will be use in LAMBDA function? help me to do...

[matszal]

Hi, Any advice how to write lambda function to publish message to iot topic?

[sanonyii]

@BaranEm Have you got lambda code for publish only. I am using loop_forever but because of this my lambda code is keep on execting. anyway i can disconnect after publish

[matszal]

@sanonyii - this code should send a message hello 2 to your topic

import boto3
import json

client = boto3.client('iot-data', region_name='eu-west-1')

pay = 'hello2'

def lambda_handler(event, context):
    response = client.publish(
        topic='mytopic/iot',
        qos=1,
        payload=pay
    )

[cyrusadkisson]

Javascript version would be like:

const AWS = require('aws-sdk');
var iotdata = new AWS.IotData({
    endpoint: 'endpointfromIOTconsole',
    apiVersion: '2015-05-28'
});

iotdata.publish({
        topic: "topic_1", // there is an aws-iot-device-sdk device-example.js you can run on a device to listen to this topic and confirm receipt
        payload: JSON.stringify({
            "message": "Hello from AWS lambda"
        })
    },
    function(err, data) {
        if (err) console.log(err, err.stack); // an error occurred
        else console.log(data); // successful response
    }
);

[bharatirkal121]

@sanonyii - this code should send a message hello 2 to your topic

import boto3
import json

client = boto3.client('iot-data', region_name='eu-west-1')

pay = 'hello2'

def lambda_handler(event, context):
    response = client.publish(
        topic='mytopic/iot',
        qos=1,
        payload=pay
    )

Hello Matszal, I am quite new to aws. Can you please share more details on this (any kind of reference) and also what is "iot-data" in boto3 parameter list you have given. Can you please explain? It would be very helpful. Thank You!

[matszal]

Hi, 'iot-data' is just the argument you pass to the constructor. You normally give the parameter name as close to the data you're dealing with. Check some more documentation, since It's been three years, since I worked with AWS. https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/boto3.html#boto3.client

[ehbb]

@fengsongAWS

The reason not recommended is because of the nature of lambda. Lambda function does not support long live connection. AWS IoT Device SDK is designed for mqtt pub/sub model with AWS IoT Cloud service.

What would be the consequence of using lamda with MQTT protocol instead of HTTP? I have a use-case to publish and also subscribe for a topic, so, I need to use aws-iot-device-sdk for MQTT.

[MohammedNoureldin]

From what I see from examples provided above, I don't understand why in python you can connect directly without specifying the endpoint, while you have to do that in JavaScript? This is odd, because fetching the IoT endpoint in cloud formation or Serverless framework is really tough (opposing to what I expected).

Can't we connect in JavaScript without specifying the endpoint just like in python?

[vick998]

`const AWS = require('aws-sdk'); var iotdata = new AWS.IotData({ endpoint: 'endpointfromIOTconsole', apiVersion: '2015-05-28' });

iotdata.publish({ topic: "topic_1", // there is an aws-iot-device-sdk device-example.js you can run on a device to listen to this topic and confirm receipt payload: JSON.stringify({ "message": "Hello from AWS lambda" }) }, function(err, data) { if (err) console.log(err, err.stack); // an error occurred else console.log(data); // successful response } );`

Tried this, and i got a null response. Any input? Thanks in advance.

[sborsay]

Make sure you have given your Lambda function IoT permissions if you want to publish. Both the AWS-SDK and Boto3 allow publishing from lambda, neither allow subscribing. Use IoT Core for incoming subscription payloads with a 'Rule' and 'rule action' connecting to your publishing Lambda from IoT Core.

Here is an example that works: https://github.com/sborsay/AWS-IoT/blob/master/IoTData_Pub_lambda.js

On Thu, Jul 28, 2022 at 6:07 AM Victor @.***> wrote:

`const AWS = require('aws-sdk'); var iotdata = new AWS.IotData({ endpoint: 'endpointfromIOTconsole', apiVersion: '2015-05-28' });

iotdata.publish({ topic: "topic_1", // there is an aws-iot-device-sdk device-example.js you can run on a device to listen to this topic and confirm receipt payload: JSON.stringify({ "message": "Hello from AWS lambda" }) }, function(err, data) { if (err) console.log(err, err.stack); // an error occurred else console.log(data); // successful response } );`

Tried this, and i got a null response. Any input? Thanks in advance.

— Reply to this email directly, view it on GitHub https://github.com/aws/aws-iot-device-sdk-js/issues/97#issuecomment-1198116585, or unsubscribe https://github.com/notifications/unsubscribe-auth/AD4KXRFKQXYVIQHV5U2CZVLVWKAYTANCNFSM4CVWNYBA . You are receiving this because you are subscribed to this thread.Message ID: @.***>