Closed datenheim closed 10 months ago
TD-er
commented
4 years ago And what about the topics per sensor? Is it enough to have variables in the topic? What should be the max. length for these topics?
I have looked at CayenneLPP when looking for LoRaWAN, and I do think it has some nice features, but it is also rather limiting.
datenheim
commented
4 years ago And what about the topics per sensor?
Good question. Many of the iot platforms have different approaches. A few seem to prefer having one topic for measurement data (data that changes often is is recorded over long periods) and another one for status info (only the last message is of interest, historical data is not kept). Most platforms seem to have one topic that contains many objects called keys or channels or what ever. For instance a JSON string with many fields. Each sensor provides one or more fields in the same topic. Thus not each sensor but the complete device has one topic.
Is it enough to have variables in the topic?
I think so. Some times configurable values (tokens) or hardware IDs are also part of the topic. If also those things (e.g. MAC, CPU-ID, System Name etc.) are available as variables it should be sufficient.
What should be the max. length for these topics?
right now I have seldom seen topics longer than 64 chars but that may not be enough for very complex scenarios of very large numbers of nodes. How constrained are we here? Already low on RAM?
I have looked at CayenneLPP when looking for LoRaWAN, and I do think it has some nice features, but it is also rather limiting.
Indeed it is limited because LoRA is extremely low bandwidth network. But it would allow to interface with LoRa devices. For instance if the ESPEasy is connected to TTN and should react on data supplied by another device that is providing data via LoRa.
The main idea here is to make the MQTT controller at least versatile enough to,... lets say connect to Thingsboard, TTN, Thingspeak, Losant, Google IOT, Amazon IOT, Azure IOT and other well known platforms that offer a Free usage plan - all with just one MQTT controller.
We should start collecting the different approaches that are used by all the platforms for authentication of devices and how data is formatted and placed into topics and keys. Since all have MQTT on board it should be possible to identify a lot of platforms that are easily reachable with on implementation. Hard thing is many platforms name things very different as they are named in the MQTT specification which is really annoying when you start to play with multiple platforms.
Thingsboard for instance authenticates with a Device ID used as MQTT Client ID and the Access Token as MQTT Username but the MQTT password is empty. Others do it completely different. Sometimes platform generated Tokens are part of topics....
If we collect the info for the platforms in one place we will see which are easy to implement and which will stay apart.
TD-er
commented
4 years ago Indeed it is limited because LoRA is extremely low bandwidth network. But it would allow to interface with LoRa devices. For instance if the ESPEasy is connected to TTN and should react on data supplied by another device that is providing data via LoRa.
Bandwidth limitation is not what I meant. It is the CayenneLPP limitation of what unit of measures can be combined as a single set. As a simple example, a GPS cannot even have an HDOP value in its dataset.
I did already add TTN integration, but only for the Microchip LoRaWAN module. Still working on the SX127x chips.
datenheim
commented
4 years ago Collecting platform data, maybe here? Everyone is welcome for additions:
datenheim
commented
4 years ago It is the CayenneLPP limitation of what unit of measures can be combined as a single set. As a simple example, a GPS cannot even have an HDOP value in its dataset.
Right. Indeed Cayenne has only described 12 data types. https://developers.mydevices.com/cayenne/docs/lora/#lora-cayenne-low-power-payload
But they reference multiple specifications from OMA and IPSO that contain far more types as can be seen here (ext.label from 3200 translates to CayennLPP type 0 and so on): http://openmobilealliance.org/wp/OMNA/LwM2M/LwM2MRegistry.html
But it is still interesting what types are present - not clear what the main purpose was. Maybe CayenneLPP is not that useful as I thought in the beginning.
TD-er
commented
4 years ago Cayenne LPP does have a new revision (not sure if released already), which does have a bit more generic approach. The main problem is that it tries to be a Swiss Army Knife for data samples, but by defining it like this, it is also very limiting. I've also thought about these low-bandwidth compression-like encodings, but it is quite hard to get a good generic format which is a good compromise between size, flexibility and also being universal. They also require it to have a fixed length for a format, regardless its content.
datenheim
commented
4 years ago Yes. I have been using the VSCP protocol for some years which originated from CAN bus and it also put a lot effort into pressing everything into the payload of 8 bytes. It started out simple and senseful but tends to logical redundancy if you reach a few hundred kinds of payloads. At that time I started to love float for everything and to always stick to the SI unit as convention - where possible. Receiving end should care for user adapted interpretation. Really did not like mangling bits and see that fixed point integer ran out out space etc. I have often signals ranging from femto- to tera-something..... Maybe the better idea is to NOT include these constraint encodings and stick with what is simple to handle und at the beat. I learned to love JSON. IMO sooner than later the message size limitations will disappear. The nodes I am playing with produce around 5MB/month JSON data and the global SIM that costs me 10€ for 500MB for 10 years is good for 8 years. So 1.3€/year. Why should I care much for compression? And there is room in JSON also for reduction of volume.
denisfrench
commented
4 years ago Just adding that this feature request has some commonality with #752, #1176, and possibly others within https://github.com/letscontrolit/ESPEasy/projects/2.
Long MQTT ClientIDs, usernames, passwords and topics seem to be getting increasingly common.
I find the OpenHab/HA client generic enough, but then have to do a custom build with increased char limits to work with Losant.
Rules and the publish command enable me to construct Losant's peculiar MQTT status messages, which is entirely doable, but this does raise the bar-of-entry for a first-time user.
tonhuisman
commented
10 months ago This seems to be resolved as Extended Credentials have been added some time ago, and the separate Client ID even longer ago. Can be closed.
Implementing a "generic MQTT controller" could open paths to many Automation/Cloud solutions at once that are not supported yet. It was touched already in #378 .
My request touches many other MQTT related topics and also the "Improve MQTT" project already started.
Domotics/OpenHAB/PiDome/Homie MQTT controllers are just pre-configured variants of the requested generic MQTT controller sadly missing one or two additional features.
For instance with Thingsboard one is only missing the possibility to define the device ID (=MQTT client ID). I know ESPEasy allows to use the Unit Name as client ID but it is limited to 25 chars instead of 36 chars required by thingsboard. Thus either increasing allowed string length for unit name or a way to configure the Client ID solves the issue. This could immediately close #1157.
If one is free to configure, use variables everywhere and not limited too much in length for:
It should allow to get ESPEasy to push data to nearly any MQTT enabled solution without too much work considering the current openHAB MQTT controller is nearly there.
Indeed the receiving side (e.g. to control ESPEasy from elsewhere via MQTT) is another topic. Here IOT solutions go very different ways....