search

TheThingsNetwork / lorawan-devices

Device Repository for LoRaWAN devices
Apache License 2.0
199 stars 385 forks source link

Vendor neutral name for DIY & small manufacturer devices #130

Closed descartes closed 3 years ago

descartes commented 3 years ago

Summary

Require a vendor neutral name for use with community devices

Why do we need this?

So DIY devices that use the compliant open source stacks like the MCCI LMiC can be selected without people ending up down the wrong path with vendors that supply components - like MCCI, RAK, Arduino etc

I'd propose that small scale manufacturers can also piggy back off the generic entries - lest you end up with few hundred entries with the same base configuration. This can include people using the various modules from vendors such as Microchip, Murara & RAK as the model.

What is already there? What do you see now?

Nothing generic

What is missing? What do you want to see?

Somethign like "Compliant Generic device based on ...." without it being an essay

At present the selection is entirely by vendor which makes sense but will be hard to signpost users as they won't really know what to select. So an additional line on the UI would be useful to tell them what to search for.

How do you propose to implement this?

I'm working on the entries, just need a reasonable name that TTI are happy with.

What's wrong with "Generic"? You've got that!

I could do "DIY" for self-build and "Based on" for those using SiPs

Can you do this yourself and submit a Pull Request?

Yes

descartes commented 3 years ago

PS, I see in the template that you are using a Vendor Id of 42 which is obviously the correct answer to any numeric query, but it's not actually allocated to anyone, so I guess is a placeholder for those paying attention.

Would it be acceptable to use 428?

johanstokking commented 3 years ago

Thanks @descartes for your suggestion.

You are specifically referring to modules and software stacks? Those would be scoped to device profiles mainly.

Or are you also referring to community devices? Like people counters, trackers, environmental sensors etc, that should include codecs etc?

Only for the latter, onboarding would become relevant, and only then a vendor ID is necessary.

descartes commented 3 years ago

I'm mostly thinking of the Arduino based LMiC stacks - so I'd be looking to reference the LMiC - some of the pain comes with the setup for older existing deployed devices on ABP (channels/frequency setup plus the RX1 delay).

At present the preferred stack, MCCI LMiC, has the RX1 window hard coded in to it as 1 second so ABP will always fail unless the source code is changed - but I'm working on getting that fixed over there.

johanstokking commented 3 years ago

OK, but then we basically need a template for the device profile, right?

descartes commented 3 years ago

Not sure I understand all the terminology - in simplistic terms I was going to create some faux-devices for the various versions of LMiC that are in the wild to aid with configuration. Probably easier for me to try a couple and then we can debug what it all means.

johanstokking commented 3 years ago

The device profile defines the boot parameters and the capabilities of the end device on the network layer. Since the device profile is strictly about the network layer, the device profile is basically tied to the LoRaWAN end device stack, its version, activation mode and band.

See https://github.com/TheThingsNetwork/lorawan-devices#end-device-profile

The way the device repository is currently organized, is that device profiles are defined in the vendor directory. So each vendor has to define their device profile.

What I think we should do is create a vendor per LoRaWAN end device stack; Mbed OS, LMiC, Basic Modem, etc, but probably just starting with LMiC. In there, we can define a generic device. Everything else follows the structure, but there wouldn't be payload codecs, data sheets and physical end device characteristics, just one or few firmware versions that represent LoRaWAN end device stack (LMiC) versions (and variants), with known bands and OTAA and ABP profiles (per band).

Let me know if I can be of any help.

TD-er commented 3 years ago

Without having read this issue, I also was confronted with this structure leading to duplicated profiles and settings and even worse it may lead to outdated settings for one device while being up-to-date for others.

Take for example the structure of the Heltec definition. They did a great job in splitting the regional profiles into separate files to avoid a lot of setting duplication. But I guess a lot of those profile settings will apply to anyone. For my (pending) PR I started with a subset of those profiles, to create a subset for the Microchip RN2483/RN2903 modules. The RN2483 only supports 433/868 MHz and the RN2903 is available in a number of versions for the 915-ish frequency band. On top of that, are those modules available in a number of firmware revisions which implement different LoRaWAN stack revisions. So for changing only a single field I already had to duplicate a lot of settings. Selecting OTAA vs. ABP also doubles the number of duplicates, etc.

I think it would be great if some parts of the settings could "inherit" from a base set, where you start with "extending" the base settings and just set differences needed for your application/region/module.

For example the RN2483 would then "extend" the settings for EU433 and EU868. An ABP configuration then only needs to "extend" ABP settings and just change whatever is different in your setup.

The same for a group of devices made by the same vendor. It is great you only have to define your codec once and simply can refer to it on a single line. But for example the "sensors" is in my use case (ESPEasy) a constant set, as the sensors directly relate to the supported plugins in my firmware, so it is as constant as the codec for all different models of hardware currently supported and that will be supported in the future. (Now RN2xx3, later SX1276, etc.)

I think the inexperienced user should only be bothered with very simple choices:

I think the first 3 are the same for any LoRaWAN device and a vendor device definition could mask some of those details for example by only stating LoRaWAN 1.0.2 is usable and/or only OTAA is implemented and it is only sold in the EU. But specific settings like frequencies, RX1 window timings, etc. are almost all the same for anyone.

The last option, about the codec may change in future revisions of my firmware. For example if support for new sensors is added, which supply data that could be sent in a very efficient way, the codec should be extended to be able to decode it. Right now, my decoder script does allow to decode all (even future) plugin data if that data is not representable in a much more efficient format. The only thing missing then is a string descibing the plugin data. For this reason, it should be really useful to be able to differentiate in codecs instead of firmware versions, or at least update the codec with a single button to fetch the latest from this repository as devices running my firmware may be updated to support new sensors. This makes it from a TTN point of view still the same device, but it only needs a single part of its definition being updated.

johanstokking commented 3 years ago

@TD-er please see https://github.com/TheThingsNetwork/lorawan-devices/issues/130#issuecomment-823534250. I'd extend that with proposing to be able to refer to another profile.

The common denominator of what you want and what has been mentioned above, is that there are devices that are using a particular LoRaWAN stack (be it a software library like LMiC or a hardware module like the RN2483), and that it would be nice to be able to refer to a standard profile.

The big problem that I want to avoid, is that fixing settings for one device leads to breaking another device. This is why definitions in the vendor folder are fully self containing, and there's no way that one can break functionality for another.

So let's say we define RN2483 version 1.0.1. It works for 10 devices. There's an 11th device that doesn't work, and suggests a fix in the profile. That breaks 5 devices. How do we deal with that? Who owns the truth of the RN2483 definition?

But specific settings like frequencies, RX1 window timings, etc. are almost all the same for anyone.

"almost" is the problem here. The settings you refer to are specific to ABP. There's really no standard way to personalize devices for a particular network. Also, I don't want to encourage device makers to use ABP by making it easy. Finally, making ABP definitions "easy" makes it also opaque and it's going to be really hard to figure out which setting is wrong. Explicit is good, especially for ABP.

TD-er commented 3 years ago

Yep, as always there is a tradeoff to be made between stability, maintainability and sharing resources. (not all are mutual exclusive of course)

Making the ultimate flexible system for maintaining these settings does take an infinite amount of time, so that's of course out of the question. Maybe we can also take a different approach, by not trying to keep the base settings in sync, but rather declare some flag where a vendor may note which base settings were used for their product. (or scan for it) This way, if there are breaking changes or bug fixes in those basic settings the maintainer can automatically get notified. A bit similar to how dependabot on GitHub does scan repositories for used libraries and notify the maintainer (or even automatically creates a PR to update the library version) when an action is required. (e.g. vulnerability update)

Also, I don't want to encourage device makers to use ABP by making it easy. Finally, making ABP definitions "easy" makes it also opaque and it's going to be really hard to figure out which setting is wrong. Explicit is good, especially for ABP.

Good point! I will think of how to formulate this in the ESPEasy documentation.