OTAA : downlink not received by node although line setClockError is added

[ma7amad97]

My current setup is as follows :

• Arduino UNO

• Dragino Shield v1.4

• Lite Gateway (programmed for chirpstack)

• Chirpstack server

Also, I am using the following pinouts :

// Pin mapping const lmic_pinmap lmic_pins = { .nss = 10, .rxtx = LMIC_UNUSED_PIN, .rst = 9, .dio = {2, 6, 7}, };

So far, I have been able to send uplinks without any problem and the server is also able to send downlinks, which is later processed by the gateway and sent to the node. Unfortunately the node does not receive it, as it keeps on printing the following

EV_TXCOMPLETE (includes waiting for RX windows) Packet qeued

In an endless loop. I literally tried everything. As well as, adding the following line in the setup as mentioned in the previous issues, and changed the 1 to several other values but nothing changed :

LMIC_setClockError(MAX_CLOCK_ERROR * 1 / 100);

Still I was not able to get any progress. Attached below is pictures illustrating the setup of the board and the shield, as well as my code. PS: I am using the latest library without editing any changes. Thank you in advance.

The used code :

#include <lmic.h>
#include <hal/hal.h>
#include <SPI.h>

// This EUI must be in little-endian format, so least-significant-byte
// first. When copying an EUI from ttnctl output, this means to reverse
// the bytes. For TTN issued EUIs the last bytes should be 0xD5, 0xB3,
// 0x70.
static const u1_t PROGMEM APPEUI[8]={ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 }; //Store data in flash (program) memory instead of SRAM. optimisation purposes
void os_getArtEui (u1_t* buf) { memcpy_P(buf, APPEUI, 8);}

// This should also be in little endian format, see above.
static const u1_t PROGMEM DEVEUI[8]={ 0xdd, 0x2c, 0x42, 0xbd, 0x7c, 0xdd, 0xd0, 0xc2 };
void os_getDevEui (u1_t* buf) { memcpy_P(buf, DEVEUI, 8);}

// This key should be in big endian format (or, since it is not really a
// number but a block of memory, endianness does not really apply). In
// practice, a key taken from ttnctl can be copied as-is.
// The key shown here is the semtech default key.
static const u1_t PROGMEM APPKEY[16] = { 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF };
void os_getDevKey (u1_t* buf) {  memcpy_P(buf, APPKEY, 16);}

static uint8_t mydata[] = "you better work!";
static osjob_t sendjob;

// Schedule TX every this many seconds (might become longer due to duty
// cycle limitations).
const unsigned TX_INTERVAL = 60;

// Pin mapping
const lmic_pinmap lmic_pins = {
    .nss = 10,
    .rxtx = LMIC_UNUSED_PIN,
    .rst = 9,
    .dio = {2, 6, 7},
};

void onEvent (ev_t ev) {
    Serial.print(os_getTime());
    Serial.print(": ");
    switch(ev) {
        case EV_SCAN_TIMEOUT:
            Serial.println(F("EV_SCAN_TIMEOUT"));
            break;
        case EV_BEACON_FOUND:
            Serial.println(F("EV_BEACON_FOUND"));
            break;
        case EV_BEACON_MISSED:
            Serial.println(F("EV_BEACON_MISSED"));
            break;
        case EV_BEACON_TRACKED:
            Serial.println(F("EV_BEACON_TRACKED"));
            break;
        case EV_JOINING:
            Serial.println(F("EV_JOINING"));
            break;
        case EV_JOINED:
            Serial.println(F("EV_JOINED"));

            // Disable link check validation (automatically enabled
            // during join, but not supported by TTN at this time).
            LMIC_setLinkCheckMode(0);
            break;
        case EV_RFU1:
            Serial.println(F("EV_RFU1"));
            break;
        case EV_JOIN_FAILED:
            Serial.println(F("EV_JOIN_FAILED"));
            break;
        case EV_REJOIN_FAILED:
            Serial.println(F("EV_REJOIN_FAILED"));
            break;
            break;
        case EV_TXCOMPLETE:
             Serial.println(LMIC.freq);
            Serial.println(F("EV_TXCOMPLETE (includes waiting for RX windows)"));
            if (LMIC.txrxFlags & TXRX_ACK)
              Serial.println(F("Received ack"));
            if (LMIC.dataLen) {
              Serial.println(F("Received "));
              Serial.println(LMIC.dataLen);
              Serial.println(F(" bytes of payload"));
            }
            // Schedule next transmission
            os_setTimedCallback(&sendjob, os_getTime()+sec2osticks(TX_INTERVAL), do_send);
            break;
        case EV_LOST_TSYNC:
            Serial.println(F("EV_LOST_TSYNC"));
            break;
        case EV_RESET:
            Serial.println(F("EV_RESET"));
            break;
        case EV_RXCOMPLETE:
            // data received in ping slot
            Serial.println(F("EV_RXCOMPLETE"));
            break;
        case EV_LINK_DEAD:
            Serial.println(F("EV_LINK_DEAD"));
            break;
        case EV_LINK_ALIVE:
            Serial.println(F("EV_LINK_ALIVE"));
            break;
         default:
            Serial.println(F("Unknown event"));
            break;
    }
}

void do_send(osjob_t* j){
    // Check if there is not a current TX/RX job running
    if (LMIC.opmode & OP_TXRXPEND) {
        Serial.println(F("OP_TXRXPEND, not sending"));
    } else {
        // Prepare upstream data transmission at the next possible time.
        LMIC_setTxData2(1, mydata, sizeof(mydata)-1, 0);
        Serial.println(F("Packet queued"));
    }
    // Next TX is scheduled after TX_COMPLETE event.
}

void setup() {
    Serial.begin(115200);
    Serial.println(F("Starting"));

    #ifdef VCC_ENABLE
    // For Pinoccio Scout boards
    pinMode(VCC_ENABLE, OUTPUT);
    digitalWrite(VCC_ENABLE, HIGH);
    delay(1000);
    #endif

     #if defined(CFG_eu868)
    LMIC_setupChannel(0, 868100000, DR_RANGE_MAP(DR_SF12, DR_SF7),  BAND_CENTI);      // g-band
    LMIC_setupChannel(1, 868300000, DR_RANGE_MAP(DR_SF12, DR_SF7B), BAND_CENTI);      // g-band
    LMIC_setupChannel(2, 868500000, DR_RANGE_MAP(DR_SF12, DR_SF7),  BAND_CENTI);      // g-band
    LMIC_setupChannel(3, 867100000, DR_RANGE_MAP(DR_SF12, DR_SF7),  BAND_CENTI);      // g-band
    LMIC_setupChannel(4, 867300000, DR_RANGE_MAP(DR_SF12, DR_SF7),  BAND_CENTI);      // g-band
    LMIC_setupChannel(5, 867500000, DR_RANGE_MAP(DR_SF12, DR_SF7),  BAND_CENTI);      // g-band
    LMIC_setupChannel(6, 867700000, DR_RANGE_MAP(DR_SF12, DR_SF7),  BAND_CENTI);      // g-band
    LMIC_setupChannel(7, 867900000, DR_RANGE_MAP(DR_SF12, DR_SF7),  BAND_CENTI);      // g-band
    LMIC_setupChannel(8, 868800000, DR_RANGE_MAP(DR_FSK,  DR_FSK),  BAND_MILLI);      // g2-band
    #elif defined(CFG_us915)
    LMIC_selectSubBand(1);
    #endif
    // LMIC init
    os_init();
    // Reset the MAC state. Session and pending data transfers will be discarded.
    LMIC_reset();
    LMIC_setClockError(MAX_CLOCK_ERROR * 1 / 100);
    // Start job (sending automatically starts OTAA too)
    do_send(&sendjob);
    Serial.println("!!!");
}

void loop() {
    os_runloop_once();
}

The gateway proof ( uplinks and downlinks ) :

and the device ( only uplinks )

[matthijskooijman]

Hard to help you out here, I'm afraid. The fact that the join works means that the machinery for downlink works properly (joinaccept is also a downlink). Maybe there's a problem with the downlink timing or modulation settings? You should probably compare the gateway radio settings to the settings used by the node (enabling verbose debug output might show the exact settings used for RX, I think).

[ma7amad97]

hey thank you for your fast reply. I changed the config.h file , specifically line 28 from #define LMIC_DEBUG_LEVEL 0 to #define LMIC_DEBUG_LEVEL 2. as well as I enabled the debug output from the settings, but nothing is happening, neither in the serial monitor or the console.

[matthijskooijman]

Did you see the comment above?

// Make sure that printf is actually
// configured (e.g. on AVR it is not by default), otherwise using it can
// cause crashing.

In particular, this means you need to enable this line too:

//#define LMIC_PRINTF_TO Serial

[ma7amad97]

I missed that, but now it works and the serial output is as follows :

continued :

continued :

The basic settings are all the same as my gateway as in Frequency, BW and SF. it doesnt mention anything regarding RX though?

Thank you

[matthijskooijman]

It actually says "Received downlink", so it is receiving something? port=-1 seems to indicate a packet without any payload. Maybe you should double-check what you're actually sending?

[ma7amad97]

Sorry, I do not quiet understand what you mean. I have not changed the code yet, I only added the lines I mentioned and my respective keys. Which means that I am sending the same “hello world” as the given example. Also am recieving it on chirpstack. But I think you are talking about something else, right? Could you explain further? Sorry for the late reply

[matthijskooijman]

Sorry, I do not quiet understand what you mean. I have not changed the code yet, I only added the lines I mentioned and my respective keys. Which means that I am sending the same “hello world” as the given example. Also am recieving it on chirpstack.

This is all uplink right, and that works? Your question is about downlink?

From the debug output, it looks like LMIC is receiving a downlink packet correctly, just without any content (no application payload and no network commands either) from what I quickly gathered from the code, which is why your sketch does not detect the dowlink (since there is no data to send). Hence my question: What downlink are you actually sending? You say the gateway sends data downlink, but if that is indeed an empty packet, then everything works as expected on the LMIC side. And if so, and you do not intend to send an empty packet, your problem will be somewhere on the server end.

Does that clarify?

[ma7amad97]

This makes perfect sense, this illustrates everything. I need to adjust it, will go through it again and let you know what I have achieved. But thank you once more for your patience and quick responses, this have been very helpful. Hope u have a great weekend! Cheers

[ma7amad97]

Hi, Sorry for the delay. Lately I have been working non-stop on trying to fix this problem. I played around with the downlink packed payload as seen below { "deviceQueueItem": { "confirmed": true, "data": "SGVsbG8=", "devEUI": "c2d0dd7cbd422cdd", "fCnt": 0, "fPort": 4 } } Which is a simple Hello. But still am not receiving anything on my end-device, although the gateway receives and sends it, any other thoughts on how to debug this? or what the problem could actually be ?

Thank you for both your time and effort!

[matthijskooijman]

That packet looks ok, assuming that data is the actual payload, not the entire packet. It also looks like a fairly high-level debug statement, is there anything logged at a lower level (i.e. that would include the entire packet, including headers, so you can verify there is actually data in there)?

One more thing you could maybe try is to look at the actual data transmission using some SDR device. LoRa packets are quite well visible on a waterfall spectrum. It's not quite trivial to software-decode them (at least I haven't managed to do so yet), but you can easily see how long the transmission lasts. If you compare two downlink messages with a short and long payload, you can see if the airtime actually changes, or if the airtime stays the same, maybe the payload is not being sent at all?

[ma7amad97]

Well, regarding it being sent or not. I have a software that sniffs each and every packet that is being received and transmitted by the gateway, as well as the gateway logs that show that the info is being sent. Both show the same thing, hence it is not the issue.

Could you perhaps show me, how would an entire packet look like ?

[matthijskooijman]

Well, regarding it being sent or not. I have a software that sniffs each and every packet that is being received and transmitted by the gateway, as well as the gateway logs that show that the info is being sent. Both show the same thing, hence it is not the issue.

You mean sniffing on the ethernet/internet side of the gateway? I was suggesting to sniff the actual radio message, to confirm that that part is actually sent correctly (to some degree, since decoding the radio packet might not be trivial).

Could you perhaps show me, how would an entire packet look like ?

I don't have a specific example, but for example see this page: http://www.techplayon.com/lora-long-range-network-architecture-protocol-architecture-and-frame-formats/

Under "Frame payload", the entire radio packet is visualized. Your "Hello" data is the "Frame payload" shown, the other stuff is added around it by the gateway forwarder software (and unpacked again by LMIC). But apparently, LMIC is receiving a packet with zero payload.

You might be able to confirm the latter by doing something like this:

--- a/src/lmic/radio.c
+++ b/src/lmic/radio.c
@@ -777,6 +777,7 @@ void radio_irq_handler (u1_t dio) {
                 now -= TABLE_GET_U2(LORA_RXDONE_FIXUP, getSf(LMIC.rps));
             }
             LMIC.rxtime = now;
+            lmic_printf("RX bytes: %u, modemconfig: 0x%x\n", readReg(LORARegRxNbBytes), readReg(LORARegModemConfig1));
             // read the PDU and inform the MAC that we received something
             LMIC.dataLen = (readReg(LORARegModemConfig1) & SX1272_MC1_IMPLICIT_HEADER_MODE_ON) ?
                 readReg(LORARegPayloadLength) : readReg(LORARegRxNbBytes);

I haven't tested this, but it should print the number of raw bytes (including all headers) received (and also the modem config, just to doublecheck that it doesn't think it is using implicit header mode somehow). This number of bytes should change when you change the payload length. If it does, then LMIC is receiving your payload, but fails to decode it somehow.

[ma7amad97]

RX bytes: 17, modemconfig: 0x72 This is what is getting printed, this means that it is receiving right?

[ma7amad97]

Also, if am sending in JSON structure, is it a problem for the library? does it only receive raw bytes?

[matthijskooijman]

Yes, but we already established that it received a packet, it just looks like an empty payload. So it receives 17 bytes, but that could be just the headers / overhead, it does not mean that there's also payload in there. I'm not directly sure how much overhead there is, but looking at the article I linked, assuming there are no frame options and assuming that the hardware takes care of the "Physical layer" overhead (which seems to be the case, looking at the decodeFrame function in LMIC), this means 13 bytes of overhead, so 4 bytes of actual payload. That suggests that:

1. Some payload is being sent
2. It's not "Hello", since that would be 5 bytes.

So maybe I miscalculated, or maybe there are frame options (and no payload) after all.

To confirm, you could dump the raw payload as received from the hardware, e.g.:

--- a/src/lmic/radio.c
+++ b/src/lmic/radio.c
@@ -784,6 +784,10 @@ void radio_irq_handler (u1_t dio) {
             writeReg(LORARegFifoAddrPtr, readReg(LORARegFifoRxCurrentAddr));
             // now read the FIFO
             readBuf(RegFifo, LMIC.frame, LMIC.dataLen);
+            lmic_printf("Packet:");
+            for (uint8_t i=0; i < LMIC.dataLen; ++i)
+                lmic_printf("%02x", LMIC.frame[i]);
+            lmic_printf("\n");
             // read rx quality parameters
             LMIC.snr  = readReg(LORARegPktSnrValue); // SNR [dB] * 4
             LMIC.rssi = readReg(LORARegPktRssiValue) - 125 + 64; // RSSI [dBm] (-196...+63)

I also suggested varying the payload length to see if this RX bytes value also changes, maybe you could also still try that?

(Also, the modemconfig shows that the implicit header mode (bitmask 0x4) is disabled, so the value printed is indeed the used number of bytes)

Also, if am sending in JSON structure, is it a problem for the library? does it only receive raw bytes?

This JSON structure should be interpreted by (the software on) the gateway and encoded into an actual LoRaWAN packet, so assuming the JSON structure matches what the gateway expects (I have no idea) then this should be fine. Unless you're saying that your payload data (e.g. your "Hello") is now a JSON structure (also fine, LMIC doesn't care about the payload, though it is probably a waste of bandwidth).

[ma7amad97]

perfect, am pretty sure we're on the right direction. Right now, I am using the server's automatic downlinks and the received payload packet is 60367eb801850100030607000166c853e1 which is interpreted below using an online decoder, it has some truth to it, as when am joining it shows that this is a join request and now it shows that the type is unconfirmed data down.

So, do u think its a decoding problem? why is the library not carrying out the rest of the task, and hanging? Thanks alot matt, I really do appreciate your help!

[matthijskooijman]

which is interpreted below using an online decoder,

Ah great, I was planning to decode by hand, but this is a lot easier :-)

From the decoded packet, it seems that those extra bytes that might have been the payload, are actually frame options (Fopts, 5 bytes). The FOpts is not decoded by your tool, but 0x03 (the first byte of FOpts) means LinkADRReq, so the gateway / server is telling the device to change datarate, channel selection and retransmission settings (the next 4 bytes of FOpts).

Even more, the packet does not even have a FPort field, and no FRMPlayload, so the hardware is really not receiving your payload. So that either means that the packet is incompletely received, or your gateway is not sending out the payload. I would suspect the latter, TBH.

Maybe you should see if your gateway software can do a low-level dump of the transmitted radio packet? Ideally, you would see the same hex dump of bytes in your gateway log as in the LMIC output. If the gateway show the same (short) packet without payload, your problem is on the gateway side. If the gateway shows a longer packet, then the problem is somewhere in the radio channel (can still be gateway or LMIC, that's hard to tell).

So, do u think its a decoding problem?

No, it's certainly not a decoding problem, since the data is simply not in the data received from the radio hardware. So this makes it either a radio transmission problem (on either side), or the gateway is not transmitting your data (would be my guess).

[ma7amad97]

So, first thing I did today was use a default gateway from the manufacturer with another chirpstack provided by the manufacturer. Now am getting other payloads, but still the online tool does not decode it properly, and the other events are also not triggered. This is the new payload : 60c5bb03ee8600000352ff00010669dfaec8. Unlinke, the previous payload, this one is 18 bytes.

[ma7amad97]

well for my application, I do not care about what the server is sending at the moment. I only need the response for the confirmation of receival, which should serve sufficient. is it possible, since we are already receiving sth, to trigger the other events after receiving?

[matthijskooijman]

The one extra byte seems to be a second frame option (0x06, meaning DevStatusReq, or a request for the device to send a device status answer with battery status and link margin).

well for my application, I do not care about what the server is sending at the moment. I only need the response for the confirmation of receival, which should serve sufficient.

It seems that this confirmation is not actually in the downlink packets you showed so far (FCtrl.ACK = false). So maybe you need to explicitly instruct your gateway / server software to generate such an ACK? Also, are you requesting an ACK in your uplink packet (that needs a parameter to SetTXData, or how the function is called exactly IIRC). This would also enable automatic retransmission of the packet by LMIC when no ACK is received.

There is of course an implicit ACK in the fact that you're receiving any packet in response, but I'm not directly sure if you can easily detect this from your sketch now (also, without requesting an ACK, the gateway / server might not even send any reply at all, in this case it seems a reply is only sent because the gateway / server wants to change some radio settings on the device, maybe not even because you instructed it to send a downlink.

If the gateway would send the ACK bit, you could easily check that in your sketch with the code you already have:

        if (LMIC.txrxFlags & TXRX_ACK)
          Serial.println(F("Received ack"));

(though re-reading the code now, it might actually be that you need to clear the TXRX_ACK and TXRX_NACK bits yourself, I haven't really tested this before so far)

[ma7amad97]

Am a total newbie. Sorry, but yes it seems that am not requesting any ACK in the uplink as the chirpstack shows that ACK = False, Is there a specific structure of the packet being sent, in order to get an acknowledgement ? Attached is my decoder on chirpstack, you can easily understand the packet am sending for the uplink from it.

var decagon_moisture_value = (bytes[0] << 8 | bytes[1] & 0xFF) / 100; var ec_value = (bytes[2] << 8 | bytes[3] & 0xFF) / 100; var decagon_temp_value = (bytes[4] << 8 | bytes[5] & 0xFF) / 100; var moist_capacitive_value = (bytes[6] << 8 | bytes[7] & 0xFF) / 100;

So normally my uplink normally looks like this 00 62 00 00 09 24 3b c4 00 00 00 00 00 00 00 00 which is before decoding SENSOR ASCII PAYLOAD: 0.99+0.00+23.4+153

[matthijskooijman]

    LMIC_setTxData2(1, mydata, sizeof(mydata)-1, 0);

The last 0 in this line means "unconfirmed uplink" (aka do not request ack). If you change that to 1, it should request an ack.

[ma7amad97]

Yes, now it is working perfectly, and am printing received ack. Thank you for your patience and time