Raw data format from Node to GW

[vdbpeter]

Hi again I'm building a MicroPython node with an RFM95W with partial success.

Can you guide me on the formatting of the Payload data, please? The payload isn't being interpreted as I expect. I looked for "data format" in your extensive documentation, but couldn't find a reference.

I'm sending this in a simple loop: while True: payload = '\!TC/88.88' print("Sending packet: \n{}\n".format(payload)) lora.println(payload)

    counter += 1
    sleep(5)

And the GW is interpreting it like this: 2020-01-26T09:42:59.185182> --- rxlora. dst=0 type=0x00 src=0 seq=0 len=10 SNR=7 RSSIpkt=-59 BW=125 CR=4/5 SF=12 2020-01-26T09:42:59.186208> 2020-01-26T09:42:59.184816 2020-01-26T09:42:59.190632> rcv ctrl pkt info (^p): 0,0,0,0,10,7,-59 2020-01-26T09:42:59.191494> splitted in: [0, 0, 0, 0, 10, 7, -59] 2020-01-26T09:42:59.192436> rawFormat(len=10 SNR=7 RSSI=-59) 2020-01-26T09:42:59.193274> rcv ctrl radio info (^r): 125,5,12,913880 2020-01-26T09:42:59.194198> splitted in: [125, 5, 12, 913880] 2020-01-26T09:42:59.195075> (BW=125 CR=5 SF=12) 2020-01-26T09:42:59.196059> rcv timestamp (^t): 2020-01-26T09:42:59.182 2020-01-26T09:42:59.196803> 2020-01-26T09:42:59.197592> adding time zone info 2020-01-26T09:42:59.198301> new rcv timestamp (^t): 2020-01-26T09:42:59.182+01:00 2020-01-26T09:42:59.198975> got first framing byte 2020-01-26T09:42:59.199668> --> got LoRa data prefix 2020-01-26T09:42:59.200326> raw format from LoRa gateway 2020-01-26T09:42:59.201008> LoRaWAN? 2020-01-26T09:42:59.202142> ['0x5c', '0x21', '0x54', '0x43', '0x2f', '0x38', '0x38', '0x2e', '0x38', '0x38'] 2020-01-26T09:42:59.203251> update ctrl pkt info (^p): 256,92,0x2F435421,11832,-3,7,-59 2020-01-26T09:42:59.204124> --> DATA encrypted: local AES not activated 2020-01-26T09:42:59.205136> --> FYI base64 of LoRaWAN frame w/MIC: XCFUQy84OC44OA== 2020-01-26T09:42:59.205931> --> number of enabled clouds is 1 2020-01-26T09:42:59.206603> --> LoRaWAN encrypted cloud[0] 2020-01-26T09:42:59.207302> uploading with python CloudTTN.py

It's not supposed to be sent to CloudTTN, but to CloudThingspeak.

The key thing to me is that at least the Text I'm sending within my Micropython node is being received correctly by the GW: '\!TC/88.88' == ['0x5c', '0x21', '0x54', '0x43', '0x2f', '0x38', '0x38', '0x2e', '0x38', '0x38']

This is encouraging.

I've spent sometime attempting to decode "post_processing_gw.py" but can't quite figure out what's missing for it to be interpreted the same way as my Arduino node (that's working rock solid).

And I couldn't find a way to print the RAW data within "post_processing_gw.py" to analyse the working Arduino node.

Or is the issue in the Header, and if so, what specifically?

Please advise.

Many thanks Peter

[vdbpeter]

Hi An update, and solved. I'd appreciate any feedback though.

I found relevant documentation in http://cpham.perso.univ-pau.fr/LORA/RPIgateway.html "3. Main Architecture".

Through careful reading of your document I found reference to the fact you intentionally (and smartly) used unix pipes between processes. This led me down the path of running lora_gateway without post processing and observing and comparing the non-printable & printable characters being decoded by the receiver. Comparing this to the documentation I was able to confirm things were working as designed, but I had to add the 4x Bytes Header(They were missing in my 1st attempt: dst=0 type=0x00 src=0 seq=0) . (see below).

I also ran into the trap "for young players" that starting Lora_gateway via the cmd.sh, Options 1 & 2 bypass the "gateway_conf.json" file leaving the Radio misconfigured. The cpp code defaults to Ch10 898MHz- whereas my radio is a 915MHz unit. I edited the cmd.sh to include the "--freq=913.88". Success.

Updated Micropython Sender script (it was tricky getting the sequence to start at 0!):

_def send(lora): counter = 0 print("LoRa Sender") dst = '\x01' type = '\x10' src = '\x0A' seq = 0

while True:
    hex_seq=(seq).to_bytes(1,'little').decode()
    # payload = 'Hello ({0})'.format(seq)
    payload = dst+type+src+hex_seq+'\!TC/88.88'
    print("Sending packet: \n{}\n".format(payload))
    lora.println(payload)

    seq +=1
    sleep(15)_

My Micropython version is based on this https://github.com/Wei1234c/SX127x_driver_for_MicroPython_on_ESP8266 (whose work was based on an Arduino implementation). It's not as extensive as your implementation, but it's a very useful start!

Peter

[CongducPham]

Hi, yes you missed the packet header. Seems that it is now working with the 4-byte header. If you need debugging, you can just run lora_gateway in standalone as you did and add --hex option so that the gateway will print what is received in hex format.

regards,

[vdbpeter]

Thanks for that tip (--hex).

I have to improve my "seq" incrementing method. It stopped at 127. Will post a reworked solution.

[vdbpeter]

Hi A question for you about the "seq" byte: technically it could count from 0 to 255, (then reset when 255).

I've found using the simple "chr(x)" works (instead of the earlier code), where x is the incrementing integer in the "while" loop. However, when the count gets to 128 Dec (\x80), the GW code seems to be misinterpreting it but on the node it's generating "\x80", "\x81" etc just fine.

You can see what's happening in the debug data:

--- rxlora. dst=1 type=0x10 src=10 seq=125 len=10 SNR=6 RSSIpkt=-43 BW=125 CR=4/5 SF=12 ^p1,16,10,125,10,6,-43 ^r125,5,12,913880 ^t2020-01-27T10:40:51.682 ▒▒5C 21 54 43 2F 38 38 2E 38 38 --- rxlora. dst=1 type=0x10 src=10 seq=126 len=10 SNR=6 RSSIpkt=-43 BW=125 CR=4/5 SF=12 ^p1,16,10,126,10,6,-43 ^r125,5,12,913880 ^t2020-01-27T10:41:07.878 ▒▒5C 21 54 43 2F 38 38 2E 38 38 --- rxlora. dst=1 type=0x10 src=10 seq=127 len=10 SNR=6 RSSIpkt=-45 BW=125 CR=4/5 SF=12 ^p1,16,10,127,10,6,-45 ^r125,5,12,913880 ^t2020-01-27T10:41:24.073 ▒▒5C 21 54 43 2F 38 38 2E 38 38 --- rxlora. dst=1 type=0x10 src=10 seq=194 len=11 SNR=6 RSSIpkt=-43 BW=125 CR=4/5 SF=12 ^p1,16,10,194,11,6,-43 ^r125,5,12,913880 ^t2020-01-27T10:41:40.270 ▒▒80 5C 21 54 43 2F 38 38 2E 38 38 --- rxlora. dst=1 type=0x10 src=10 seq=194 len=11 SNR=6 RSSIpkt=-43 BW=125 CR=4/5 SF=12 ^p1,16,10,194,11,6,-43 ^r125,5,12,913880 ^t2020-01-27T10:41:56.467 ▒▒81 5C 21 54 43 2F 38 38 2E 38 38 --- rxlora. dst=1 type=0x10 src=10 seq=194 len=11 SNR=7 RSSIpkt=-45 BW=125 CR=4/5 SF=12 ^p1,16,10,194,11,7,-45 ^r125,5,12,913880 ^t2020-01-27T10:42:12.664 ▒▒82 5C 21 54 43 2F 38 38 2E 38 38 --- rxlora. dst=1 type=0x10 src=10 seq=194 len=11 SNR=6 RSSIpkt=-45 BW=125 CR=4/5 SF=12 ^p1,16,10,194,11,6,-45 ^r125,5,12,913880 ^t2020-01-27T10:42:28.860 ▒▒83 5C 21 54 43 2F 38 38 2E 38 38

Can you offer some guidance here please?

[CongducPham]

chr() return a character, so not all value are possible.

[vdbpeter]

Problem solved. For completion, and hopefully help others, here's how I solved it:

Found that using 'bytearray()" did the trick. The 'payload' was being encoded (str.encode()) downstream, prepping the data for transmission, and was messing up the header bytes with values >7f (changing them to 2 byte values). Concatenating the bytearray with the encoded 4x header bytesstring kept the byte integrity I was after:

Code fragment of "LoRaSender.py", part of the "SX127x_driver_for_MicroPython_on_ESP8266" : dst = 1 type = 16 src = 1 seq = 126

while True:
    print('LoRaSender Sequence#: ',seq)
    string='\!TC/88.88'
    payload =  bytearray([dst,type,src,seq]) + string.encode()
    lora.println(payload)
    seq +=1
    if seq == 256:
        seq = 0
    sleep(60)

---

Being sent: [Memory - free: 15872 allocated: 22080] LoRaSender Sequence#: 130 Sending packet: bytearray(b'\x01\x10\x01\x82\!TC/88.88')

From sx127x.println.write: bytearray(b'\x01\x10\x01\x82\!TC/88.88')

Being Received: --- rxlora. dst=1 type=0x10 src=1 seq=130 len=10 SNR=6 RSSIpkt=-55 BW=125 CR=4/5 SF=12 ^p1,16,1,130,10,6,-55 ^r125,5,12,913880 ^t2020-02-01T05:59:20.778 ▒▒5C 21 54 43 2F 38 38 2E 38 38

[CongducPham]

Great to hear! All the best for your project.

[vdbpeter]

Thanks- and thanks again for your work!!