Use wildcards in Autodiscovery patterns?

[sammyke007]

Is it possible to use wildcards in the Autodiscovery patterns? I've tried *, # or %, but none of them work? My fulltopics are %prefix%/tasmota/*room*/%topic%/. I can configure each room in the pattern match, but using a wildcard could be a lot easier (like %prefix%/tasmota/*/%topic%/).

[jziolkowski]

pushed fix for missing trailing /

[barbudor]

Hello @jziolkowski Any progress on this one ?

[jziolkowski]

yes-ish :D I got frustrated with it so I spent some time refactoring and repackaging the project to have complete up-to date automatic release automation. I will rebase the discovery branch on top of that new layout and back to work.

However I've decided to postpone native tasmota discovery, as the feature creep is huge as is. So I'll make the wildcards work, and will tackle native disco later. Stay tuned ;)

[jziolkowski]

@sammyke007 @barbudor I've just pushed a few big commits. Please test.

I've removed native discovery (separate task)

Reworked 90% of message processing (some places might be still missing, it's WIP) but most should work.

Refactored a lot. Cool stuff.

[jziolkowski]

@sammyke007 any updates? I'd appreciate some response

[sammyke007]

I'll test tomorrow. I've been very busy with a big project, sorry...

[sammyke007]

Ok... tomorrow was 3 weeks ago, but it's still incredibly busy at work. I've tried running the latest commit, but things have changed. python tdmgr.py is not working anymore, is it .\tdmgr\run.py now?

[jziolkowski]

Yes. Or you can do

pip install .

and then do

tdmgr

Should work

[sammyke007]

That's easier, tnx. Still the autodiscovery branch?

Default config, only broker set up:

2024-02-08 21:46:39 [INFO] ### TDM START ###
QWidget::insertAction: Attempt to insert null action
QWidget::insertAction: Attempt to insert null action
QWidget::insertAction: Attempt to insert null action
QWidget::insertAction: Attempt to insert null action
QWidget::insertAction: Attempt to insert null action
QWidget::insertAction: Attempt to insert null action
QWidget::insertAction: Attempt to insert null action
QWidget::insertAction: Attempt to insert null action
2024-02-08 21:46:40 [INFO] MQTT: Subscribed to tasmota/discovery/+/config, tele/#, stat/#, +/tele/#, +/stat/#
2024-02-08 21:46:40 [INFO] DISCOVERY(LEGACY): LWT from an unknown device tele/tasmota/zolder_3dprinter/LWT
2024-02-08 21:46:40 [INFO] DISCOVERY(LEGACY): LWT from an unknown device tele/tasmota/badkamer_main/LWT
2024-02-08 21:46:40 [INFO] DISCOVERY(LEGACY): LWT from an unknown device tele/tasmota/badkamer_spiegel/LWT
2024-02-08 21:46:40 [INFO] DISCOVERY(LEGACY): LWT from an unknown device tele/tasmota/badkamer_ventilatie/LWT
2024-02-08 21:46:40 [INFO] DISCOVERY(LEGACY): LWT from an unknown device tele/tasmota/berging_main/LWT
2024-02-08 21:46:40 [INFO] DISCOVERY(LEGACY): LWT from an unknown device tele/tasmota/overloop_zolder_main/LWT
2024-02-08 21:46:40 [INFO] DISCOVERY(LEGACY): LWT from an unknown device tele/tasmota/dressing_main/LWT
2024-02-08 21:46:40 [INFO] DISCOVERY(LEGACY): LWT from an unknown device tele/tasmota/living_eetkamer_vast/LWT
2024-02-08 21:46:40 [INFO] DISCOVERY(LEGACY): LWT from an unknown device tele/tasmota/garage_main/LWT
2024-02-08 21:46:40 [INFO] DISCOVERY(LEGACY): LWT from an unknown device tele/tasmota/slk_fran_main/LWT
2024-02-08 21:46:40 [INFO] DISCOVERY(LEGACY): LWT from an unknown device tele/tasmota/garage_bovenste_stekkerblok/LWT
2024-02-08 21:46:40 [INFO] DISCOVERY(LEGACY): LWT from an unknown device tele/tasmota/keuken_aanrecht/LWT
2024-02-08 21:46:40 [INFO] DISCOVERY(LEGACY): LWT from an unknown device tele/tasmota/garage_poort/LWT
2024-02-08 21:46:40 [INFO] DISCOVERY(LEGACY): LWT from an unknown device tele/tasmota/berging_server/LWT
2024-02-08 21:46:40 [INFO] DISCOVERY(LEGACY): LWT from an unknown device tele/tasmota/slk_ouders_main/LWT
2024-02-08 21:46:40 [INFO] DISCOVERY(LEGACY): LWT from an unknown device tele/tasmota/slk_vic_main/LWT
2024-02-08 21:46:40 [INFO] DISCOVERY(LEGACY): LWT from an unknown device tele/tasmota/tuin_achterdeur/LWT
2024-02-08 21:46:40 [INFO] DISCOVERY(LEGACY): LWT from an unknown device tele/tasmota/tuin_schuifraam/LWT
2024-02-08 21:46:40 [INFO] DISCOVERY(LEGACY): LWT from an unknown device tele/tasmota/tuin_tuinhuis_extern/LWT
2024-02-08 21:46:40 [INFO] DISCOVERY(LEGACY): LWT from an unknown device tele/tasmota/wc_main/LWT
2024-02-08 21:46:40 [INFO] DISCOVERY(LEGACY): LWT from an unknown device tele/tasmota/tuin_zijgevel/LWT
2024-02-08 21:46:40 [INFO] DISCOVERY(LEGACY): LWT from an unknown device tele/tasmota/zolder_main/LWT
2024-02-08 21:46:40 [INFO] DISCOVERY(LEGACY): LWT from an unknown device tele/tasmota/keuken_microgolf/LWT
2024-02-08 21:46:40 [INFO] DISCOVERY(LEGACY): LWT from an unknown device tele/tasmota/keuken_oven/LWT
2024-02-08 21:46:40 [INFO] DISCOVERY(LEGACY): LWT from an unknown device tele/tasmota/keuken_koelkast/LWT
2024-02-08 21:46:40 [INFO] DISCOVERY(LEGACY): LWT from an unknown device tele/tasmota/keuken_vaatwasser/LWT
2024-02-08 21:46:40 [INFO] DISCOVERY(LEGACY): LWT from an unknown device tele/tasmota/garage_wasmachine/LWT

[jziolkowski]

Ok, then in your autodiscovery patterns dialog you need to add %prefix%/tasmota/%topic%/ or %prefix%/+/%topic%/ and it should pick the devices. Or whatever else pattern you also use that needed the wildcard in the first place.

[sammyke007]

I've added %prefix%/tasmota/%topic%/ indeed. It started discovering devices but crashed with:

2024-02-08 22:00:27 [INFO] DISCOVERY(LEGACY): Discovered topic=garage_bovenste_stekkerblok with fulltopic=%prefix%/tasmota/%topic%/
2024-02-08 22:00:27 [INFO] DISCOVERY(LEGACY): Discovered topic=tuin_achterdeur with fulltopic=%prefix%/tasmota/%topic%/
2024-02-08 22:00:27 [INFO] DISCOVERY(LEGACY): Discovered topic=badkamer_spiegel with fulltopic=%prefix%/tasmota/%topic%/2024-02-08 22:00:27 [INFO] DISCOVERY(LEGACY): Discovered topic=tuin_zijgevel with fulltopic=%prefix%/tasmota/%topic%/
2024-02-08 22:00:27 [INFO] DISCOVERY(LEGACY): Discovered topic=garage_poort with fulltopic=%prefix%/tasmota/%topic%/
2024-02-08 22:00:27 [INFO] DISCOVERY(LEGACY): Discovered topic=garage_wasmachine with fulltopic=%prefix%/tasmota/%topic%/
2024-02-08 22:00:27 [INFO] DISCOVERY(LEGACY): Discovered topic=keuken_microgolf with fulltopic=%prefix%/tasmota/%topic%/2024-02-08 22:00:27 [INFO] DISCOVERY(LEGACY): Discovered topic=berging_main with fulltopic=%prefix%/tasmota/%topic%/
2024-02-08 22:00:27 [INFO] DISCOVERY(LEGACY): Discovered topic=slk_ouders_main with fulltopic=%prefix%/tasmota/%topic%/
2024-02-08 22:00:27 [INFO] DISCOVERY(LEGACY): Discovered topic=dressing_main with fulltopic=%prefix%/tasmota/%topic%/
2024-02-08 22:00:27 [INFO] DISCOVERY(LEGACY): Discovered topic=keuken_koelkast with fulltopic=%prefix%/tasmota/%topic%/
2024-02-08 22:00:27 [INFO] DISCOVERY(LEGACY): Discovered topic=zolder_3dprinter with fulltopic=%prefix%/tasmota/%topic%/2024-02-08 22:00:27 [INFO] DISCOVERY(LEGACY): Discovered topic=tuin_tuinhuis_extern with fulltopic=%prefix%/tasmota/%topic%/
2024-02-08 22:00:27 [INFO] DISCOVERY(LEGACY): Discovered topic=slk_fran_main with fulltopic=%prefix%/tasmota/%topic%/
2024-02-08 22:00:27 [INFO] DISCOVERY(LEGACY): Discovered topic=zolder_main with fulltopic=%prefix%/tasmota/%topic%/
2024-02-08 22:00:27 [INFO] DISCOVERY(LEGACY): Discovered topic=badkamer_main with fulltopic=%prefix%/tasmota/%topic%/
2024-02-08 22:00:27 [INFO] DISCOVERY(LEGACY): Discovered topic=overloop_zolder_main with fulltopic=%prefix%/tasmota/%topic%/
2024-02-08 22:00:27 [INFO] DISCOVERY(LEGACY): Discovered topic=berging_server with fulltopic=%prefix%/tasmota/%topic%/
2024-02-08 22:00:27 [INFO] DISCOVERY(LEGACY): Discovered topic=garage_main with fulltopic=%prefix%/tasmota/%topic%/
2024-02-08 22:00:27 [INFO] DISCOVERY(LEGACY): Discovered topic=tuin_schuifraam with fulltopic=%prefix%/tasmota/%topic%/
2024-02-08 22:00:27 [INFO] DISCOVERY(LEGACY): Discovered topic=keuken_vaatwasser with fulltopic=%prefix%/tasmota/%topic%/
2024-02-08 22:00:27 [INFO] DISCOVERY(LEGACY): Discovered topic=badkamer_ventilatie with fulltopic=%prefix%/tasmota/%topic%/
2024-02-08 22:00:27 [INFO] DISCOVERY(LEGACY): Discovered topic=keuken_aanrecht with fulltopic=%prefix%/tasmota/%topic%/
2024-02-08 22:00:27 [INFO] DISCOVERY(LEGACY): Discovered topic=wc_main with fulltopic=%prefix%/tasmota/%topic%/
2024-02-08 22:00:27 [INFO] DISCOVERY(LEGACY): Discovered topic=keuken_oven with fulltopic=%prefix%/tasmota/%topic%/
2024-02-08 22:00:27 [INFO] DISCOVERY(LEGACY): Discovered topic=living_eetkamer_vast with fulltopic=%prefix%/tasmota/%topic%/
2024-02-08 22:00:28 [INFO] DISCOVERY(LEGACY): Discovered topic=slk_vic_main with fulltopic=%prefix%/tasmota/%topic%/
Traceback (most recent call last):
  File "C:\Users\verdo\AppData\Local\Programs\Python\Python312\Lib\site-packages\tdmgr\GUI\dialogs\main.py", line 391, in mqtt_message
    device.process_message(msg)
  File "C:\Users\verdo\AppData\Local\Programs\Python\Python312\Lib\site-packages\tdmgr\util\__init__.py", line 306, in process_message
    self.process_sensor(msg.payload)
  File "C:\Users\verdo\AppData\Local\Programs\Python\Python312\Lib\site-packages\tdmgr\util\__init__.py", line 289, in process_sensor
    sensor_data = json.loads(payload)
                  ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\verdo\AppData\Local\Programs\Python\Python312\Lib\json\__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\verdo\AppData\Local\Programs\Python\Python312\Lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\verdo\AppData\Local\Programs\Python\Python312\Lib\json\decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
               ^^^^^^^^^^^^^^^^^^^^^^
json.decoder.JSONDecodeError: Expecting ':' delimiter: line 1 column 100 (char 99)

[sammyke007]

Second run it crashed on: json.decoder.JSONDecodeError: Invalid control character at: line 1 column 44 (char 43)

Third run is running without error.

[jziolkowski]

Weird, it should not crash like that. Does the logfile show what was the error payload?

[jziolkowski]

Also, does all your devices have this tasmota part, or do you have multiple different types? Because if not, then you didn't need wildcards in the first place :D

[sammyke007]

I was planning to use %prefix%/tasmota/"room" (wildcard)/%topic%/ so it would look like %prefix%/tasmota/+/%topic%/, but because that would raise problems with TDM, I've included the room names in the topics...

[jziolkowski]

any more crashes?

[sammyke007]

Seems to be stable over here!

[jziolkowski]

Thanks. I'll do some final touches then, merge and release

[C2Leu]

+1 to the above feature

[C2Leu]

@jziolkowski any chance of a new release? thanks