[Feature Request]: Serial Module TXTMSG mode should support sending to specified Channel or Destination Node

[aaro668]

Platform

NRF52, ESP32

Description

I would like to set up a mesh network of sensor nodes in a forest setting to measure custom sensors. Each node will be battery powered with solar charging, so would need to be one of the low power Meshtastic boards (Rak), and I plan to have a separate MCU (ATMEGA328p barebones or Arduino Pro Mini) as a sensor interface to get sensor values at a set interval and send to the Meshtastic device using Serial Module, to be forwarded onto the mesh, and ideally it would be addressed directly to a gateway node that is connected to MQTT, from MQTT we can deal with the data as we please. I was hoping to use a small low power secondary MCU for the sensor interface that will spend most of its time in some form of sleep mode to conserve power. I tried loading the Meshtastic-Arduino library (Arduino client) onto Arduino Pro Mini, but I got an error that there is not enough memory on the Pro Mini. It seems that TXTMSG mode is much simpler to implement and has very low memory requirement, with the draw-back that messages are just sent as broadcast on default channel. This might be ok in some cases, but in my particular case the mesh might have 20-30 of these sensor nodes as well as 2-3 regular messaging nodes connected to Android client etc. You can imagine if each sensor broadcast values on default channel on the mesh how this would spam the regular client-connected nodes, where if the messages were DM'd to the gateway it would keep things much quieter. I am not a software developer so I am not sure exactly how this could be done, but I imagined that something like : : "text message to send" would maybe work. If was not present it would default to 0, if was not present it would default to BROADCAST_ADDRESS. Like I said I am not a software developer at all, and if there might be a different way to implement this I am open to idea's!

[caveman99]

Problem is the 2 modes exist because of the lightweight simplicity of the one mode (textmsg, almost transparent layer of transport) and the protobuf interface to have full control over sending. Adding complexity to the textmsg mode would break other applications. My recommendation would be to use the protobuf mode and get a bit beefier MCU.

Unless we get a PR to review i don't see anybody working on this, sorry.

[thebentern]

The only middle ground I could see is perhaps accepting a json structure with the destination and channel fields and a payload along side these, but that would be a pretty breaking change for this mode. IMO it would need to be an altogether new mode

[caveman99]

The only middle ground I could see is perhaps accepting a json structure with the destination and channel fields and a payload along side these, but that would be a pretty breaking change for this mode. IMO it would need to be an altogether new mode

Agreed, minus the suggestes use of JSON. The existing json code in the firmware doesn't work well already. I don't think we should roll new features based on it.

[aaro668]

+1 for the new JSON mode in Serial Modul!

JSON mode would at minimum need a "payload" property, using default channel and BROADCAST_ADDRESS Optional properties used by Meshtastic would be & Payload could be a text string, or an object with properties of its own, like payload.temperature or payload.voltage Example JSON to illustrate: { "destination": 2086361396, "channel": 0, "payload": { "temp": 25.7, "voltage": 3.47, "count": 10 } }

This could become a very interesting feature in conjunction with MQTT and Node-red or Home Assistant for the front end in telemetry applications where the second MCU is required to read custom sensors (anything is possible here) or where CPU intensive calculations are needed (averaging etc) ......

Ack....... just saw @caveman99 replied downvoting the JSON, oh well, it was worth thinking about for a bit there :)

[caveman99]

Ack....... just saw @caveman99 replied downvoting the JSON, oh well, it was worth thinking about for a bit there :)

Pull requests are always accepted. Thing is, the current implementation in our firmware of the JSON encoder/decoder is not very robust. And we swapped that lib out once already. So that pull request would need to swap that out again (no, don't just use ArduinoJson, it is huge!)

If you can pull off a serial JSON mode based on a lightweight and robust Json lib, we'd be all over it :-)

[thebentern]

@aaro668 we've not had good luck with json in the project, and we sadly don't have the flash to spare for larger libraries like ArduinoJson which are considerably large.

Perhaps some delimited value scheme instead, but the crux of the issue is that this mode largely exists because "protobufs are hard"? At this point, I almost feed like there would be more value in some scripts to use as a translation layer from json to protobufs, so that folks could have the ergonomics of json and the functionality of native protobufs. That's a bit outside of the scope of this project though :-)

[aaro668]

So(correct me if I am wrong): Protobufs are used because of efficiency of radio on-air time? JSON is resource intensive (CPU, RAM) and thus makes it an in-efficient way to send data over radio?

@thebentern when you say "translation layer from json to protobufs" I assume you mean this would be on the secondary MCU and the front end consuming the MQTT? So that json is translated to protobufs on secondary mcu, and then back to json again in front end Node-Red or HA?

[garthvh]

Protobufs are light weight for sending over the wire, JSON needs to be done somewhere else, not on the device.