Take mqtt parameters from the command line (or environment)

It would be helpful to not have to hard code MQTT parameters in the script.

You could probably copy a bit of code from rtl_433's Home Assistant auto discovery script

https://github.com/merbanan/rtl_433/blob/master/examples/rtl_433_mqtt_hass.py

It uses argparse rather than getopt.

    parser = argparse.ArgumentParser(formatter_class=argparse.RawDescriptionHelpFormatter,
                                     description=AP_DESCRIPTION,
                                     epilog=AP_EPILOG)

    parser.add_argument("-d", "--debug", action="store_true")
    parser.add_argument("-q", "--quiet", action="store_true")
    parser.add_argument("-u", "--user", type=str, help="MQTT username")
    parser.add_argument("-P", "--password", type=str, help="MQTT password")
    parser.add_argument("-H", "--host", type=str, default="127.0.0.1",
                        help="MQTT hostname to connect to (default: %(default)s)")
    parser.add_argument("-p", "--port", type=int, default=1883,
                        help="MQTT port (default: %(default)s)")
    parser.add_argument("-c", "--ca_cert", type=str, help="MQTT TLS CA certificate path")
    parser.add_argument("-r", "--retain", action="store_true")
    parser.add_argument("-f", "--force_update", action="store_true",
                        help="Append 'force_update = true' to all configs.")
    parser.add_argument("-R", "--rtl-topic", type=str,
                        default="rtl_433/+/events",
                        dest="rtl_topic",
                        help="rtl_433 MQTT event topic to subscribe to (default: %(default)s)")
    parser.add_argument("-D", "--discovery-prefix", type=str,
                        dest="discovery_prefix",
                        default="homeassistant",
                        help="Home Assistant MQTT topic prefix (default: %(default)s)")
    parser.add_argument("-i", "--interval", type=int,
                        dest="discovery_interval",
                        default=600,
                        help="Interval to republish config topics in seconds (default: %(default)d)")
    parser.add_argument("-x", "--expire-after", type=int,
                        dest="expire_after",
                        help="Number of seconds with no updates after which the sensor becomes unavailable")
    parser.add_argument("-I", "--ids", type=int, nargs="+",
                        help="ID's of devices that will be discovered (omit for all)")
    args = parser.parse_args()

I'm still trying to understand how these programs might be used by others.

I had been assuming that someone might want to analyze or monitor devices in their neighborhood, using an rtl_433/RTL_SDR setup on some computer on their network to do the monitoring and one or several computers on the same network to analyze or monitor. In that use case, I'd think it would be easiest to set up the parameters once, in the source file, and then just run it. I'd think that would be just as easy as adding the parameters on a command line that they might set up in a shell script -- maybe easier where I provided the sed command to edit the files.

But if some number of users would want to run the programs against some number of different rtl_433/RTL-SDR hosts, then the flexibility of command-line parameters would make sense.

Do you have a sense of which way most people might use these?

Taking command line arguments (IMHO) is one of the most common conventions if not the most.

Yes, if the arguments get too long, users put that in a quick and dirty shell script. That's how I run rtl_433 and I think I'm probably in the majority. rtl_433 does take a config file, but there seems to have been enough confusion on the correct grammar over the years that I have avoided it.

For many reasons, configuration should not be part of source code. Especially if you use source control and expect the software to be used by others. It's a good way to leak private information when you accidentally commit the source with your configs.

If you don't want to parse command line parameters, and/or want to avoid the need for a wrapper script, use environment variables. The script mentioned above takes environment variables as well, but command line arguments will take precedence.

    # allow setting MQTT username and password via environment variables
    if not args.user and 'MQTT_USERNAME' in os.environ:
        args.user = os.environ['MQTT_USERNAME']

    if not args.password and 'MQTT_PASSWORD' in os.environ:
        args.password = os.environ['MQTT_PASSWORD']

As for how I'd be using stats collection, I have two MQTT brokers (one in each house). Each of those is fed by 2-3 rtl_433 instances, 2 on 433mhz in different areas of the house for better coverage and the 3rd one is for 345mhz. So in reality I have 5 rtl_433 instances to monitor rf reception for. I may soon add a 6th since my neighbor has a weather station on 915mhz, that I'd like to compare to my own in near real time.

I have on the order of 20-30 devices decoded by rtl_433 at each location - temperature, humidity, acurite weather station, freezer temperatures, security sensors (window/door, motion, smoke/co2), water/leak detectors.

This what fits on the screen with rtl_watch maximized. Everything in rtl_watch is extra large so I'm not fitting much on screen. I'm not sure if that's a high DPI issue because I'm using 4K monitors. (Curiously when rtl_watch gets in a broken state from what I described in #3 - the window shrinks to what I would describe as normal window size.)

Hope this helps.

This probably doesn't belong here, but as an FYI here's the print summary output after running for a while.

____________________________________________________________________

rtl_watch: Printing summary of recorded rtl_433 packets

First entry recorded at:  2023-02-15 11:05:02.542401
Last entry recorded at:   2023-03-07 18:19:00.992055
Processed  2248  de-duplicated records of a total of  17328  records

Device                     Rec Cnt     Mean ±  𝜎        Min      Max
Acurite-515 1783                57      9.2 ± 1.74      7.1     18.3
Acurite-Atlas 924              403     12.7 ± 2.13      8.7     25.5
Acurite-Tower 12053            219     12.9 ± 2.35      8.6     23.3
Acurite-Tower 14322            303     12.7 ± 2.05      7.6     23.0
Acurite-Tower 2633             206     12.6 ± 1.98      8.0     24.8
Acurite-Tower 9884             229      8.3 ± 2.26      4.6     18.6
Ambientweather-F007TH 115       47     13.2 ± 2.54     11.4     22.6
Ambientweather-F007TH 139       57     12.3 ± 0.57     11.3     13.9
Ambientweather-F007TH 155       74     11.6 ± 3.08      6.7     22.6
Ambientweather-F007TH 166       55     12.8 ± 2.61     11.3     22.9
Ambientweather-F007TH 192       57     12.7 ± 2.13     10.0     23.3
Ambientweather-F007TH 218       50     12.3 ± 1.76      8.4     22.9
Ambientweather-F007TH 241       57     11.8 ± 1.67     10.0     22.5
Ambientweather-F007TH 42        57     12.7 ± 2.36     11.0     23.4
Ambientweather-F007TH 64        69     12.3 ± 1.77      7.1     21.3
Generic-Remote 32768             1      9.9 ± 0.00      9.9      9.9
Generic-Remote 520               1      9.5 ± 0.00      9.5      9.5
Honeywell-Security 637081        1      9.6 ± 0.00      9.6      9.6
Thermopro-TP11 1080            305     12.1 ± 0.56      9.1     13.9

____________________________________________________________________

Before you scratch your head over the first record being Feb 15th, I have the 345mhz rtl_433 instance use the MQTT retained flag. So there is an old message that the broker retained from then that it sends first on new connections:

{"time":"2023-02-15 11:05:02.542401","model":"Honeywell-Security","id":637081,"channel":8,"event":4,"state":"closed","contact_open":0,"reed_open":0,"alarm":0,"tamper":0,"battery_ok":1,"heartbeat":1,"mic":"CRC","mod":"ASK","freq":345.06608,"rssi":-0.103813,"snr":9.5754,"noise":-9.67921}
{"time":"2023-03-07 18:25:29.668169","model":"Ambientweather007TH","id":155,"channel":1,"battery_ok":1,"temperature_F":73.6,"humidity":20,"mic":"CRC","mod":"ASK","freq":433.89392,"rssi":-3.58941,"snr":18.95045,"noise":-22.5399}

These were both very helpful in giving me a sense of how the programs might be used elsewhere.

There are now a couple of situations where I think I want command-line parameters to be able to override default internal settings, so I'll be investing a little effort to incorporate them and will add mqtt parameters to the list. I'd been concerned about the accidental exposure of internal information -- the issue you raised -- and see that cmdline parameters would be a way to avoid that. I'm not a fan of envt variable approach, but it looks easy enough to add so I'll give it a shot.

It was also helpful to see your mean SNR's. Generally lower than the ones I see from my own sensors or close neighbors here, but reliable enough to provide service. And I can see why the inter-transmission gap times, number of packets/device, and number of transmissions/device could be useful to help assess device stability/service reliability. I'll try to work the code into something useful for you.

I've incorporated both command-line and environment-parameters into the parameter-setting for rtl_watch. If the "host" value is not specified in some other manner, rtl_watch prompts the user and defaults the other values to what I think are standard values, so only "-h " is needed on the command line (or specified in environment).

The edits in rtl_watch may currently be only in the v2.0.0 (development) branch, which you can download, but I hope to merge that into the main branch shortly.

Rob,

Command-line arguments and environment variables are both implemented in the v2.0.0 branch of rtl_watch. I hope to merge that branch into main shortly, after a few more additions to the code.

Regarding the font size / screen size issue, rtl_watch now has vertical and horizontal scroll bars. I've put more device information per row (frequency, inter-transmission gap times, packets per transmission), but you can select which ones of those you want to see and so reduce the display width.

But I notice that there are some differences between the Mac and Raspbian tkinter handling of the screen. rtl_watch dynamically changes the font size when you toggle between window and full screen, and that seems not to be consistent among devices. I'll rethink the font-sizing process again to see if I can make it more flexible for individual user needs. Probably not in v2.0.0 but in a subsequent revision.

Thanks, again, for the feedback ... I think it helped me improve the functionality of all three programs (rtl_watch, rtl_433_stats, and DNT).

David

On 3/7/23 5:50 PM, rct wrote:

Taking command line arguments (IMHO) is one of the most common conventions if not the most.

Yes, if the arguments get too long, users put that in a quick and dirty shell script. That's how I run rtl_433 and I think I'm probably in the majority. rtl_433 does take a config file, but there seems to have been enough confusion on the correct grammar over the years that I have avoided it.

For many reasons, configuration should not be part of source code. Especially if you use source control and expect the software to be used by others. It's a good way to leak private information when you accidentally commit the source with your configs.

If you don't want to parse command line parameters, and/or want to avoid the need for a wrapper script, use environment variables. The script mentioned above takes environment variables as well, but command line arguments will take precedence.

|# allow setting MQTT username and password via environment variables if not args.user and 'MQTT_USERNAME' in os.environ: args.user = os.environ['MQTT_USERNAME'] if not args.password and 'MQTT_PASSWORD' in os.environ: args.password = os.environ['MQTT_PASSWORD'] |

As for how I'd be using stats collection, I have two MQTT brokers (one in each house). Each of those is fed by 2-3 rtl_433 instances, 2 on 433mhz in different areas of the house for better coverage and the 3rd one is for 345mhz. So in reality I have 5 rtl_433 instances to monitor rf reception for. I may soon add a 6th since my neighbor has a weather station on 915mhz, that I'd like to compare to my own in near real time.

I have on the order of 20-30 devices decoded by rtl_433 at each location - temperature, humidity, acurite weather station, freezer temperatures, security sensors (window/door, motion, smoke/co2), water/leak detectors.

This what fits on the screen with rtl_watch maximized. Everything in rtl_watch is extra large so I'm not fitting much on screen. I'm not sure if that's a high DPI issue because I'm using 4K monitors. (Curiously when rtl_watch gets in a broken state from what I described in #3 https://github.com/hdtodd/rtl_watch/issues/3 - the window shrinks to what I would describe as normal window size.)

image https://user-images.githubusercontent.com/38770/223569979-89a7921b-dd3f-4f24-9bab-c14ee9aa4279.png

Hope this helps.

— Reply to this email directly, view it on GitHub https://github.com/hdtodd/rtl_watch/issues/2#issuecomment-1458986987, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJWBXAGBN4RMGKG3RP6WHTW263RRANCNFSM6AAAAAAVSYFKZM. You are receiving this because you commented.Message ID: @.***>

hdtodd / rtl_watch

Take mqtt parameters from the command line (or environment) #2