search

gotthardp / lorawan-server

Compact server for private LoRaWAN networks
https://gotthardp.github.io/lorawan-server
MIT License
956 stars 329 forks source link

unable to connect to AWS IoT core and see device messages #482

Closed rnukala closed 6 years ago

rnukala commented 6 years ago

Hi gothardp,

I'm trying to integrate the server to aws IoT core using the connectorr as you have described in the documentation. I have also looked at other issue up on the forum and ensured that I am using right clientId and mqtts.

My issues are:

  1. I am not connecting to aws iot core. Attached is the error from debug.log image

I have added handler with payload as cayenne lpp , created connector with url "mqtts://aws-endpoint:8883", In received frames section, I can see the device messages. However, on the "Events" page this the corresponding error for the received frame.

image

  1. I am unclear about what should I be fiiling in for " Publish Uplinks", "Subscribe", "Received Topic". For now these are the settings i have used

image

my gateway is multitech configured in packet forwarder mode, I usually have to subscribe to "lora/+/up" to see the device message. So am unclear as to, what do I configure in the above fields and what topic do I need to subscribe on my aws iot core.

Could you please help me with these issues Thank you

gotthardp commented 6 years ago

Hello. 1) You need to define a Backend Handler and set its name in the Device Profile to the Application field. 2) The Received Topic is a pattern for parsing the subscribed topic. It is OK to leave the Received Topic empty.

rnukala commented 6 years ago

@gotthardp thank you for the quick response, I have set the device profile to handler. But I still see mqtt reconnecting to the aws iot core and never connected or send messages. Just to double check with you again

  1. mqtt url : mqtts://endpoint:8883 (do i need to mention the port?)
  2. client id is my aws account id ( correct ? or should i give clientId thing name, well I have tried both) 3.I have the policy set in. 4.publish uplinks: out/{devaddr}/ (do i need to keep it devaddr or should i use actuall device address)

Is there anything you think I could be missing.

Thank you

gotthardp commented 6 years ago

It is safer to mention the port. For the uplink topic, keep using the {devaddr} template (do not replace it). Please also make sure you have "devaddr" in your Uplink Fields. I'd recommend removing the trailing slash from the publish topic (use out/{devaddr} only).

rnukala commented 6 years ago

@gotthardp added the devaddrs in uplink and removed the / from publish link. It is still showing reconnecting issue image

let me share screenshots

image image image image

I can't seem to understand what else I am missing. Thank you and appreciate your support.

gotthardp commented 6 years ago

As you correctly said before: the "client id" shall be your aws account id.

rnukala commented 6 years ago

@gotthardp what should I be seeing in the logs when it gets connected? Does it say connected to aws iot core?

gotthardp commented 6 years ago

With the release 0.6.0 you should see a lot of [Client ] messages, including "connected with".

rnukala commented 6 years ago

@gotthardp ,

In the debug.log it is still showing me " recconnecting , connector xxxx, reconnecting".

In authentication Client Id = awsaccount id, Auth ="username+password", Name = blank, password key = blank, user certificate = provided , private key = provide. So name and password should be blank? The certs are generated when i created device in aws iot core so that why I thought i should use client id as my things name, either case I should leave username and password blank?

Is there an end device that you could please suggest for testing the flow till aws iot core. I'm asking this because the messages from end device are seen as unknown frame.

Thank you,

gotthardp commented 6 years ago

AWS has a debugging log, which may tell you more.

I checked the two "unknown" frames you have above in the log and it's not according to LoRaWAN standard. I suspect your gateway may be doing some decoding. Are you sure it is in the packet_forwarder mode?

rnukala commented 6 years ago

Yes it is in packet forwarder mode, the other thing which I have noticed is I have an end device which seem to send proper data. I could also see sent messages in the received frame section. The configuration in device profile for that device was default semtech-mote.

But then when for iot core I changed the application to the new handler that was created. Now I cannot seem to see messages in the received frame.

Is it possible that if the messages are not in received frame you never get anything published out to Aws iot core .

Thank you,

On Wed, 12 Sep 2018, 1:07 pm Petr Gotthard, notifications@github.com wrote:

AWS has a debugging log, which may tell you more.

I checked the two "unknown" frames you have above in the log and it's not according to LoRaWAN standard. I suspect your gateway may be doing some decoding. Are you sure it is in the packet_forwarder mode?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gotthardp/lorawan-server/issues/482#issuecomment-420624283, or mute the thread https://github.com/notifications/unsubscribe-auth/ALmr68rYBCTeh6B-5QCHrQndNTwpW3Pyks5uaPkagaJpZM4WhSv7 .

gotthardp commented 6 years ago

What is forwarded is also logged. If nothing is logged, nothing was sent. Could you please set the Profile Application to the name of your Handler, send few frames and share with me the debug.log?

rnukala commented 6 years ago

@gotthardp attached is the debug.log , look for device "00EF2B20"

I sent a few frames from the device, then subscribe to topics "out/00EF2B20" on aws iot core, I did not see any messages. Also, when the application profile is "semtech-mote", i can see value in "Data" column. But when I changed it into "testApp" which is my handler, i cant see any data value (in screenshot).

Received frames: image

event log image

handler image

debug.log

Thank you

gotthardp commented 6 years ago

Which Erlang version are you using? Erlang 21?

rnukala commented 6 years ago

@gotthardp sorry for late answer , yes it is Erlang OTP 21

gotthardp commented 6 years ago

You are using an old server version (0.5.x?), which is not compatible with Erlang 21. Therefore the errors you have in the logs. Either downgrade Erlang to 19 or 20, or upgrade the server to 0.6.0.

rnukala commented 6 years ago

@gotthardp i am sorry, should have checked version from the beginning following your note.

I upgraded the version to 0.6.0 , added groups and areas to I can now see message "connected to mqtt" in the logs phewwww. Thank you.

However, on aws iot core i tried to subscribe to topic out/# also I cant seem to receive any message. Would that be correct? I will only be able to test the same device "00EF2B20" on monday and update you. Also for one of the end device I only see missing uplinks message, it doesnot display my handler in "Application column" on events log either.

image

For the device "019C2501" , I have specified my own handler as I have mentioned it to in previous discussions.

image

Please let me know your thoughts on this. Thank you again .

gotthardp commented 6 years ago

One step closer :) The log shows that AWS closes the connection just after you try to subscribe. It usually does that when it dislikes the topic. Are you sure about your lora/+/up? Could you try something simpler like in/# which is what I tested?

rnukala commented 6 years ago

@gotthardp hurray it is working, I'm finnaly able to connect to aws iot core and can publish messages.

  1. I changed the subscribe to in/# ,
  2. topic to which I should subscribe on aws iot core is out/{devaddr},
  3. I also had to make changes in "Payload" in handler to custom one or empty, because when I had Cayenne LPP I could not receive messages.
  4. I had to change client id to my aws account id (which was a bit confusing until I could see published messages)

Best thing the sensor with which I started to use your application long back (older post) also worked. I do have one of the sensor giving missed_uplinks error, I will to look into the issue.

Thank you very much for your full support and quick responses :).

gotthardp commented 6 years ago

Cool. Do you want to investigate the "Cayenne LPP" issue too? If your device is sending the LPP format, it shall work.

rnukala commented 6 years ago

@gotthardp yes I will have to go through the doc for the format and will try configuring device to send it in LPP format.

rnukala commented 6 years ago

@gotthardp the device with missed_uplink, I can see couple of messages getting published , my handler is configured to decode payload as ascii (was trying)

2018-09-17 14:39:34.795 [info] <0.9445.1> [xxxxxxxxx@xxxxx:43013] SENT: PUBLISH(Q1, R0, D0, TopicName=out/019C2501, PacketId=23, Payload=<<123,34,100,97,116,97,34,58,34, 52,56,66,50,54,65,70,55,66,55, 49,52,66,65,65,49,56,54,49,54, 65,52,51,69,51,66,68,66,57,65, 57,49,69,67,48,56,66,56,50,53, 66,67,50,48,50,51,53,70,54,57, 49,55,53,70,69,48,49,56,50,50, 49,55,50,50,69,56,48,48,67,55, 55,55,57,54,56,69,69,65,65,51, 66,69,52,66,69,70,66,53,50,69, 34,44,34,100,97,116,101,116,105, 109,101,34,58,34,50,48,49,56,45, 48,57,45,49,55,84,49,52,58,51, 57,58,51,52,90,34,44,34,100,101, 118,97,100,100,114,34,58,34,48, 49,57,67,50,53,48,49,34,44,34, 112,111,114,116,34,58,49,54,56, 44,34,116,101,120,116,34,58,34, 72,239,191,189,106,239,191,189, 92,117,48,48,49,52,239,191,189, 239,191,189,239,191,189,92,117, 48,48,49,54,239,191,189,62,59, 219,154,239,191,189,239,191,189, 92,98,239,191,189,37,239,191, 189,32,35,95,105,92,117,48,48, 49,55,95,239,191,189,92,117,48, 48,49,56,92,34,92,117,48,48,49, 55,92,34,239,191,189,92,117,48, 48,48,48,239,191,189,119,239, 191,189,239,191,189,234,163,190, 75,239,191,189,46,34,125>>) 2018-09-17 14:39:34.797 [info] <0.9445.1> [xxxxxx@xxxxxxx:43013] SENT: <<50,173,2,0,12,111,117,116,47,48,49,57,67,50,53,48,49,0,23,123,34,100,97,116,97,34,58,34,52,56,66,50,54,65,70,55,66,55,49,52,66,65,65,49,56,54,49,54,65,52,51,69,51,66,68,66,57,65,57,49,69,67,48,56,66,56,50,53,66,67,50,48,50,51,53,70,54,57,49,55,53,70,69,48,49,56,50,50,49,55,50,50,69,56,48,48,67,55,55,55,57,54,56,69,69,65,65,51,66,69,52,66,69,70,66,53,50,69,34,44,34,100,97,116,101,116,105,109,101,34,58,34,50,48,49,56,45,48,57,45,49,55,84,49,52,58,51,57,58,51,52,90,34,44,34,100,101,118,97,100,100,114,34,58,34,48,49,57,67,50,53,48,49,34,44,34,112,111,114,116,34,58,49,54,56,44,34,116,101,120,116,34,58,34,72,239,191,189,106,239,191,189,92,117,48,48,49,52,239,191,189,239,191,189,239,191,189,92,117,48,48,49,54,239,191,189,62,59,219,154,239,191,189,239,191,189,92,98,239,191,189,37,239,191,189,32,35,95,105,92,117,48,48,49,55,95,239,191,189,92,117,48,48,49,56,92,34,92,117,48,48,49,55,92,34,239,191,189,92,117,48,48,48,48,239,191,189,119,239,191,189,239,191,189,234,163,190,75,239,191,189,46,34,125>>

the sensor is sending either comma separated decimal values or just hex values. Either way these values seem meaningless to me,
Please let me know your thoughts, do you have like a hex decoder ? I'm new to erlang. If I do not specify Payload in the handler, am I supposed to see raw message in "data"?

Thank you,

gotthardp commented 6 years ago

If you don't specify Payload and include data into the Uplink Fields, you get data as hexa-string.

What you posted here is MQTT message. The numbers are dec-bytes and probably useless.

gotthardp commented 6 years ago

The data sent in the MQTT are {"data":"48B26AF7B714BAA18616A43E3BDB9A91EC08B825BC20235F69175FE018221722E800C777968EEAA3BE4BEFB52E","datetime":"2018-09-17T14:39:34Z","devaddr":"019C2501","port":168,"text":"..."}. The text field includes a binary chaos which is likely a reason why AWS doesn't like it. You did set Payload to ASCII, but your message apparently doesn't include ASCII.

rnukala commented 6 years ago

ok, I will check with hex, A quick question, for every connector it takes one handler, can i write code to handle payload of all 10 different sensors (each having different payload format ) in just one handler? Or do I need to maintain one handler for each device. In that case do i need to add one connector per device?

Thank you

gotthardp commented 6 years ago

Yes, one handler can handle multiple formats. There is one small example in https://github.com/gotthardp/lorawan-server/blob/master/doc/Handlers.md#parse-uplink

gotthardp commented 6 years ago

Any update, please? Shall I close the issue?

rnukala commented 6 years ago

No sorry didn't get a chance to look into erlang code for that particular device, yes please do close the issue and my sincere thanks to you for all the support.

On Tue, 25 Sep 2018, 10:52 am Petr Gotthard, notifications@github.com wrote:

Any update, please? Shall I close the issue?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gotthardp/lorawan-server/issues/482#issuecomment-424279063, or mute the thread https://github.com/notifications/unsubscribe-auth/ALmr6x-fwEYm4aGV0m7iL9n6xksa11fKks5uefzfgaJpZM4WhSv7 .