merbanan / rtl_433

Program to decode radio transmissions from devices on the ISM bands (and other frequencies)
GNU General Public License v2.0
6.03k stars 1.31k forks source link

housecode or id "mapping" ? #2540

Closed devZer0 closed 11 months ago

devZer0 commented 1 year ago

hello,

i have the problem (and so do others) that when changing batteries, the houscode or id of a sensor changes.

when processing the data from rtl_433 with scripts, i can easily add a mapping file to turn changing id's/housecodes into static names, which i need to adjust after battery swap. that can be easily done.

but what can i do when using influx as output option?

what about adding some id/housecode mapping feature into rtl_433 itself ?

merbanan commented 1 year ago

The issue here is addressing. When the id is random you can look at the model or even maybe the rssi if you set the receiver to a fixed gain. We do have some translation logic so maybe we will accept a patch for that but the main aim is to touch the data as little as possible.

gdt commented 1 year ago

I personally would like to see a written design for how the larger ecosystem of receiving, logging and using should work, not only influxdb and MQTT but also Home Assistant. The id mapping issue is sensible but difficult, and there is also deduplication, not only among one rtl_433 instance but across instances on multiple machines. (I am actually running two computers with 4 dongles, with each having a 433 sensor, because at the moment I'm trying to deal with a difficult device on 433.)

Overall, I lean to rtl_433 heading to just producing json (and human readable for testing) and having other programs do dedup, id mapping, and conversion to MQTT, influxdb, etc. Yes, these certainly could be in the binary by linking, but I think it's best if they are maximally detangled. I have found the "send json to 1433 localhost" to work well and all my reliability issues have been the receiving program not doing the right thing or the dongle flaking.

ProfBoc75 commented 1 year ago

Hi,

@gdt : agree with you, rtl_433 should serve as a flat pass-through, all other applications will do the job, mqtt, node red, and so on. From my side, I'm using 2 dongles linked to Domoticz and rtl_433 plugin. In case an ID was changed, it's simple at Domoticz level, you just have to replace the old by the new discovered device, with builtin feature "replace" at the sensor level, this will keep all the metric history. Yes, it's manual, but not often and very simple in very few clicks.

devZer0 commented 1 year ago

@gdt https://github.com/gdt : agree with you, rtl_433 should serve as a flat pass-through, all other applications will do the job, mqtt, node red, and so on.

but you can hardly automate that in all those applications

if using rtl_433 with influxdb, the measurements will getting written with a different id, so after some battery swaps, you won't have one device with one id in your database but one device with multiple different id's

there is no way to get "in between" besides implementing your own influx db writer tool/shellscript

so, why handle something at the higher levels, when it's probably not that hard to implement at the low level by a simple mapping file

, mechanism or maybe some mapping param(s)? i can even think of some helper script which automates the process of re-mapping the device id when being changed, as empty battery is typically a one-by-one process and there is also the battery-was-swapped flag with many sensors.  so we could detect that and automatically remap with some intelligence in a helper script.  that would be a lot of work to implement that on top of every data consumer application. if you have setup some monitoring with rtl433 and database / graphical app it's easy for a techical person the remap the sensor id.  but what if the monitoring is running at a remote location without internet and the person changing battery is non technical/skilled person ? roland Am 24.06.23 um 00:34 schrieb Bruno OCTAU: > > Hi, > > @gdt : agree with you, rtl_433 should serve > as a flat pass-through, all other applications will do the job, mqtt, > node red, and so on. > From my side, I'm using 2 dongles linked to Domoticz and rtl_433 > plugin. In case an ID was changed, it's simple at Domoticz level, you > just have to replace the old by the new discovered device, with > builtin feature "replace" at the sensor level, this will keep all the > metric history. Yes, it's manual, but not often and very simple in > very few clicks. > > — > Reply to this email directly, view it on GitHub > , > or unsubscribe > . > You are receiving this because you authored the thread.Message ID: > ***@***.***> >
zuckschwerdt commented 1 year ago

There will always be advanced use-cases we can't reasonably cater to. Take every output, esp. MQTT, InfluxDB as a base option for the standard use-case. In all other cases use a bridge script to read, process, and forward data.

We don't want to be the gatekeepers on what integration and transformation is possible, we would like to provide an easy way to process unwrapped but mostly raw radio data with your own mechanism. See e.g. https://github.com/merbanan/rtl_433/blob/master/examples/rtl_433_influxdb_relay.py

gdt commented 1 year ago

As @zuckschwerdt said, if you want something more complicated to land in your influxdb, then use a script that processes json output on the way. A more complicated script with remapping support would surely be welcome as an example, or it could be its own project. And as I said, I lean to rtl_433 having direct output soley as 1) human readable for debugging and 2) json (pipe or socket) for scripts/etc. to process to whatever. However, I do not expect my leaning to lead to removal of influxdb/mqtt and csv from the codebase.

devZer0 commented 1 year ago

ok, got it -and i can live with that without a problem. i just was interested if others would find it useful,too.

in terms of unix philosophy, it's always best to have every tool do it's special purpose and chain them together via pipe to do more special things....

roland

Am 24.06.23 um 15:50 schrieb Greg Troxel:

As @zuckschwerdt https://github.com/zuckschwerdt said, if you want something more complicated to land in your influxdb, then use a script that processes json output on the way. A more complicated script with remapping support would surely be welcome as an example, or it could be its own project. And as I said, I lean to rtl_433 having direct output soley as 1) human readable for debugging and 2) json (pipe or socket) for scripts/etc. to process to whatever. However, I do not expect my leaning to lead to removal of influxdb/mqtt and csv from the codebase.

— Reply to this email directly, view it on GitHub https://github.com/merbanan/rtl_433/issues/2540#issuecomment-1605514253, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA2X7OKJ6XFRT22IBJH3YP3XM3WB7ANCNFSM6AAAAAAZP22NUI. You are receiving this because you authored the thread.Message ID: @.***>

gdt commented 11 months ago

I'm calling this a dup of #2635