ABP: What endianness should I use for the FILLMEIN fields?

sbrl commented 5 years ago

In the OTAA example sketches, it helpfully instructs you as to what endianness you should use for each item.

Unfortunately, the same cannot be said for the ABP example. After extensive testing I'm unable to get OTAA working with TTN, so I (and some other people I know) are trying ABP instead while we try to work out what's going wrong with OTAA.

It says this:

// LoRaWAN NwkSKey, network session key
static const PROGMEM u1_t NWKSKEY[16] = { FILLMEIN };

// LoRaWAN AppSKey, application session key
static const u1_t PROGMEM APPSKEY[16] = { FILLMEIN };

// LoRaWAN end-device address (DevAddr)
// See http://thethingsnetwork.org/wiki/AddressSpace
// The library converts the address to network byte order as needed.
static const u4_t DEVADDR = FILLMEIN; // <-- Change this address for every node!

In particular, should NWKSKEY and APPSKEY be in big endian or little endian? I'm confused.

hsuomal commented 5 years ago

My personal code has these comments, all with most significant byte first // LoRaWAN NwkSKey, network session key MSB first byte order // LoRaWAN AppSKey, application session key, MSB first byte order // LoRaWAN end-device address (DevAddr), MSB first

I think OTAA code had something in the other order too.

sbrl commented 5 years ago

Ah, thanks @hsuomal! Yeah, the OTAA one has 2 things in lsb and 1 in msb.

Perhaps the example sketch could be updated to clarify this?

terrillmoore commented 5 years ago

Thanks for your comments and your good group effort on this.

Feel free to submit a pull request. I don't like ABP -- it's not LoRaWAN conformant unless your device stores the uplink count and downlink count in NVRAM and never resets them -- so I never use or recommend use of these sketches to new users. If no pull request is forthcoming, I'll update them in the next push.

By the way, I have no problem with OTAA with TTN; I use that exclusively. If OTAA isn't working, downlink isn't working, and if downlink isn't working, the device can't be managed by the network. So ABP is, in my view, not something worth investing much effort in.

Perhaps you should investigate your OTAA problems further, because although the LMIC has limitations, OTAA support is not one of them.

sbrl commented 5 years ago

Thanks for the reply, @terrillmoore.

What do you mean my "managed by the network"? What kind of things does the network do?

I've heard that with OTAA you can't turn the device off, which is unfortunately critical to my use-case in order to save enough power.

Unfortunately, I'm completely baffled as to the problems with OTAA, with no clear way to debug them. Currently, I know these things:

I'm using an Uno with a Dragino LoRa Shield
LMIC emits EV_TXSTART and the join start event
LMIC fails to emit EV_TXCOMPLETE
The gateway (Rak831, Semtech packet forwarder) receives the message, but claims it fails the CRC check
I've forced the gateway to forward the message anyway, and TTN doesn't even recognise it

To this end, I'm out of leads. Considering this is a Msc-level project for University, I've got limited time to dig into the specifics of why OTAA isn't working. While this is certainly something I can look into in my own time (I've got another personal project that could potentially benefit from fixing it), I'm forced to use ABP here to keep the project on schedule.

If OTAA isn't working, downlink isn't working

In a class A device with ABP I get EV_TXCOMPLETE (including RX windows). Doesn't this mean that downlink does infact work on ABP?

I'll see what I can do about a PR.

terrillmoore commented 5 years ago

In a class A device with ABP I get EV_TXCOMPLETE (including RX windows). Doesn't this mean that downlink does infact work on ABP?

No, not at all. It must means that nothing was received during the downlink window. The LMIC doesn't emit a EV_TXCOMPLETE during join cycles. HEAD (not an official release) emits EV_JOIN_TXCOMPLETE when a join cycle completes without a join, but the current release doesn't yet have that fix.

with OTAA you can't turn the device off.

That's not true at all. You have to save the same state that you must save for ABP. (If you turn off an ABP device, and don't save the FcntUp and FcntDown, you will lose sync with the network. As long as you're having to do that, it's trivial to save the OTAA-generated NwkSKey and AppSKey.) Since the NwkSKey and AppKey change rarely, they don't add any complexity. Saving the FcntUp and FcntDown adds a lot of complexity to avoid wearing out EEPROM; but that's no different between the two cases. Anything that works for ABP in this situation will also work for OTAA.

What do you mean my "managed by the network"? What kind of things does the network do?

The network:

sends you the real list of channels it wants you to use. With ABP, you really have no way of knowing unless you dig deeply into the network documentation; and you have to hope they don't change.
tells you how loud it thinks you should be talking and at what data rate. This is critical for managing network connection.

I admit I'm a bit of a telecoms and standards geek, but a class A device that doesn't have working downlink really is problematic in my mind, and so I'd rather the open-source community doesn't get a bad reputation by shrugging issues off.

I've never seen a problem with uplink CRC. All uplink messages are supposed to have a CRC attached. I'm not sure how you could even cause that to happen. Transmit uses the exact same path for join and for data. It sound as if you might be trying to send before the join completes (because transmit uses the same buffer as join, and this could corrupt things while the radio is still sending the message). What regional plan are you using? It would be interesting to find out what is going on. I can think of a few possibilities:

the bad CRC message is spurious; the millis()/micros() time in your board is inaccurate. This will cause the receive window to be opened too late. We have seen that board support packages are somewhat inadequate in this area sometimes; LMIC is a "hard real-time" (as opposed to "soft real time") subsystem, and we have found that this reveals subtle bugs in the platform timekeeping. (This is one reason we maintain our own BSPs.) You can often work around this by adjusting the clock rate using LMIC_setClockError(), e.g.,
```
void setup() {
     // ....
     // set clock error to 5%
     LMIC_setClockError(5 * MAX_CLOCK_ERROR / 100);
     // ...
}
```
there is a bug in the code for your region which has been fixed on HEAD. I tend to run HEAD, and I can tell from your description that you're using an older version. Try updating.
there's something configured wrong in the pinmap for the Dragino shield / Uno combination.

I understand about limited resources, so of course, you have to do what you must under the circumstances. I'm just giving context for my decisions.

Best regards, --Terry

sbrl commented 5 years ago

Hey, @terrillmoore!

Ah, I see. That makes things a lot clearer. If I use the latest master and compile & flash the OTAA example, I get this in the serial monitor:

Packet queued
222: EV_JOINING
480: Unknown event
420683: Unknown event
441599: Unknown event
840480: Unknown event
1030564: Unknown event
1450843: Unknown event
1459338: Unknown event
1858156: Unknown event
1920568: Unknown event
2340847: Unknown event
2365823: Unknown event
2764705: Unknown event

I get this when compiling:


Build options changed, rebuilding all
/home/sbrl/Arduino/libraries/arduino-lmic/src/hal/getpinmap_thisboard.cpp: In function 'const Arduino_LMIC::HalPinmap_t* Arduino_LMIC::GetPinmap_ThisBoard()':
/home/sbrl/Arduino/libraries/arduino-lmic/src/hal/getpinmap_thisboard.cpp:55:72: note: #pragma message: Board not supported -- use an explicit pinmap
         #pragma message("Board not supported -- use an explicit pinmap")
                                                                        ^
/home/sbrl/Arduino/libraries/arduino-lmic/src/lmic/lmic.h:519:1: warning: type of 'LMIC' does not match original declaration
 DECLARE_LMIC; //!< \internal
 ^
/home/sbrl/Arduino/libraries/arduino-lmic/src/lmic/lmic.c:39:1: note: previously declared here
 DEFINE_LMIC;
 ^
Sketch uses 22516 bytes (69%) of program storage space. Maximum is 32256 bytes.
Global variables use 1569 bytes (76%) of dynamic memory, leaving 479 bytes for local variables. Maximum is 2048 bytes.
Low memory available, stability problems may occur.

....even with this defined at the top of the sketch:

#define DISABLE_PING
#define DISABLE_BEACONS

...and that LMIC_setClockError() call directly below LMIC_reset().

The low memory warning is somewhat concerning, because this isn't even with any other code or libraries loaded (e.g. TinyGPS++ and SD).

Here's the pin mapping:


const lmic_pinmap lmic_pins = {
    .nss = 10,
    .rxtx = LMIC_UNUSED_PIN,
    .rst = 9,
    .dio = {2, 6, 7},
};

I've checked it like 8 times - and it worked when using ABP with this fork, but doesn't appear to work with the fork you maintain: https://github.com/mcci-catena/arduino-lmic

It transmits packets, the gateway receives them.... but then I get CRC_FAIL: 100.00% in the logs of the semtech packet forwarder:

RF packets received by concentrator: 1
# CRC_OK: 0.00%, CRC_FAIL: 100.00%, NO_CRC: 0.00%

What regional plan are you using?

I'm in the UK - I should be using 868MHz. I haven't actually checked LMIC's configuration - I thought it was supposed to be configured for the EU by default.

sbrl commented 5 years ago

Update on that, TTN sees just a single packet in the gateway traffic monitor after leaving it running for a bit. Details of that:

Payload: 80C4C51281570BEF59FEDE78BC7FBEFFF7A3B9AC3F5CD375FC470AA438FB3139A650FF922BEB37A7318284C61472382C3293119061DF968C69EEA2603AB665B83B1C99D7F764D061ADD4B49282F1479BED8753D64AA95B4624735B5CB258181ED656BF21F4C63F8AF4502439B8B4CEC05132F5C7BA7B6E458E6AA6D0A3133248C819014108E434DC303EE2FB3D3D5DA11D91BC0D6996D8C1F1F85DD5FD7F69C300F839B24532B2B9735FD6157B778FCA338E55EA19E1C8129AC7FEFA39E9D1FF12509AAE85D97EA3921D02FDBCA85931FA99893D008777C1C84134E5B0580B8708D5218831C1E855F1EB50246A

0msbridge "ttn-bridge-eu" receive

    backend:packet-forwarder

0.03msbridge "ttn-bridge-eu" forward

    backend:ttn

2.36msrouter ttn-router-eu receive

    gateway:eui-b827ebfffee68b6e

2.52msrouter ttn-router-eu build downlink

    options:2

2.54msrouter ttn-router-eu drop

    reason:no brokers

Also, here's a picture of the device in question.

mcci-catena / arduino-lmic

ABP: What endianness should I use for the FILLMEIN fields? #330