Koenkk / zigbee2mqtt

Zigbee 🐝 to MQTT bridge 🌉, get rid of your proprietary Zigbee bridges 🔨
https://www.zigbee2mqtt.io
GNU General Public License v3.0
11.78k stars 1.64k forks source link

Bunch of WARNINGS after update to 1.25.0 #12040

Closed Upgreydd closed 2 years ago

Upgreydd commented 2 years ago

What happened?

I've updated to 1.25.0 (HA installation supervised)

and this appear in log:

22-04-02 18:36:07 WARNING (MainThread) [supervisor.addons.options] Unknown option 'base_topic' for Zigbee2mqtt (45df7312_zigbee2mqtt)
22-04-02 18:36:07 WARNING (MainThread) [supervisor.addons.options] Option 'external_converters' does not exist in the schema for Zigbee2mqtt (45df7312_zigbee2mqtt)
22-04-02 18:36:07 WARNING (MainThread) [supervisor.addons.options] Option 'devices' does not exist in the schema for Zigbee2mqtt (45df7312_zigbee2mqtt)
22-04-02 18:36:07 WARNING (MainThread) [supervisor.addons.options] Option 'groups' does not exist in the schema for Zigbee2mqtt (45df7312_zigbee2mqtt)
22-04-02 18:36:07 WARNING (MainThread) [supervisor.addons.options] Option 'homeassistant' does not exist in the schema for Zigbee2mqtt (45df7312_zigbee2mqtt)
22-04-02 18:36:07 WARNING (MainThread) [supervisor.addons.options] Option 'permit_join' does not exist in the schema for Zigbee2mqtt (45df7312_zigbee2mqtt)
22-04-02 18:36:07 WARNING (MainThread) [supervisor.addons.options] Option 'advanced' does not exist in the schema for Zigbee2mqtt (45df7312_zigbee2mqtt)
22-04-02 18:36:07 WARNING (MainThread) [supervisor.addons.options] Option 'device_options' does not exist in the schema for Zigbee2mqtt (45df7312_zigbee2mqtt)
22-04-02 18:36:07 WARNING (MainThread) [supervisor.addons.options] Option 'blocklist' does not exist in the schema for Zigbee2mqtt (45df7312_zigbee2mqtt)
22-04-02 18:36:07 WARNING (MainThread) [supervisor.addons.options] Option 'passlist' does not exist in the schema for Zigbee2mqtt (45df7312_zigbee2mqtt)
22-04-02 18:36:07 WARNING (MainThread) [supervisor.addons.options] Option 'queue' does not exist in the schema for Zigbee2mqtt (45df7312_zigbee2mqtt)
22-04-02 18:36:07 WARNING (MainThread) [supervisor.addons.options] Option 'frontend' does not exist in the schema for Zigbee2mqtt (45df7312_zigbee2mqtt)
22-04-02 18:36:07 WARNING (MainThread) [supervisor.addons.options] Option 'experimental' does not exist in the schema for Zigbee2mqtt (45df7312_zigbee2mqtt)
22-04-02 18:36:07 WARNING (MainThread) [supervisor.addons.options] Option 'availability' does not exist in the schema for Zigbee2mqtt (45df7312_zigbee2mqtt)
22-04-02 18:38:16 INFO (SyncWorker_3) [supervisor.docker.interface] Stopping addon_45df7312_zigbee2mqtt application

What did you expect to happen?

No errors? Replaced my configuration to:

data_path: /config/zigbee2mqtt
socat:
  enabled: false
  master: pty,raw,echo=0,link=/tmp/ttyZ2M,mode=777
  slave: tcp-listen:8485,keepalive,nodelay,reuseaddr,keepidle=1,keepintvl=1,keepcnt=5
  options: '-d -d'
  log: false
mqtt:
  server: mqtt://core-mosquitto:1883
serial:
  port: >-
    /dev/serial/by-id/usb-ITead_Sonoff_Zigbee_3.0_USB_Dongle_Plus_...

And errors gone, but where I supposed to add config now?

How to reproduce it (minimal and precise)

Just update with custom configuration

Zigbee2MQTT version

1.25.0

Adapter firmware version

20211217

Adapter

SOnOff zibee plus stick

Debug log

No response

Koenkk commented 2 years ago

See first point of https://github.com/zigbee2mqtt/hassio-zigbee2mqtt/releases/tag/v1.25.0-1, configuration can now be done via the frontend.

smulle48 commented 2 years ago

@Upgreydd the config is moved to the configuration.yaml file in the zigbe2mqtt folder

MNLFDF commented 2 years ago

@Koenkk Having the same problem and don’t exactly know what to do. “Config can be done via the frontend”. Doesn’t this happen automatically when updating to 1.25.x? If not, what should I put where? I do locate the configuration.yaml file at the zigbee2mqtt folder, but there is the same configuration as when I look at “configuration, supervisor, zigbee2mqtt, config. I reinstalled the 1.24 backup and now all errors are gone again. So please write me (and maybe other none coders) precisely what to do and where. Thanks in advance Koen! (For you it’s easy to understand, but not for people like me). 😂

fliphess commented 2 years ago

Would it be possible to use the secrets.yaml from home assistant and the secrets mechanism? I almost committed my MQTT passwords to my public git repository because of this change, which is a bit suboptimal :)

smulle48 commented 2 years ago

@Koenkk Having the same problem and don’t exactly know what to do. “Config can be done via the frontend”. Doesn’t this happen automatically when updating to 1.25.x? If not, what should I put where? I do locate the configuration.yaml file at the zigbee2mqtt folder, but there is the same configuration as when I look at “configuration, supervisor, zigbee2mqtt, config. I reinstalled the 1.24 backup and now all errors are gone again. So please write me (and maybe other none coders) precisely what to do and where. Thanks in advance Koen! (For you it’s easy to understand, but not for people like me). 😂

My "old" config looks like this (after the update):

Skærmbillede 2022-04-03 kl  22 05 24

The rest of the config has to be done in the configuration.yaml file that's located in the zigbee2mqtt folder. It looks like this: Skærmbillede 2022-04-03 kl  22 07 40

MNLFDF commented 2 years ago

@smulle48 Thanks for your info. With your info I managed to change both configurations and get it working again with Zigbee2mqtt version 1.25.1

mcarty2 commented 2 years ago

I guess everything is configured now in the frontend, under 'settings'. I was getting a lot of warnings like those, but everything was working. Looks like all my old settings form the old configuration.yaml were imported to the frontend.

z2mqtt

Philou06 commented 2 years ago

Hi, for me, to solve this with an existing installation, in Add-ons / zigbee2mqtt / Configuration choose Reset configuration, check with File Editor the file zigbee2mqtt/configuration.yaml, if you see your previous options configuration, restart Host and warnings disappears

malosaa commented 2 years ago

well i clicked the wrong reset defaults hehe

working now

sanderkoenders commented 2 years ago

Also clicked the wrong reset button at first. Just to clarify, you need to hit the three dots in the right corner. The option to reset your configuration is there.

cmonteiro128 commented 2 years ago

Hmm still getting the warning message

Update: seems to have worked now

Shponzo commented 2 years ago

When I do this, I lose all my connected devices. Does anyone has this problem too?

malosaa commented 2 years ago

When I do this, I lose all my connected devices. Does anyone has this problem too?

What exactly did you do ? I did not lose all my devices at all.

Did you delete the yaml file ? Don't do that at all, just only reset the config in Z2M

regards

Shponzo commented 2 years ago

with File Editor the file zigbee2mqtt/configuration.yaml

When I click reset, it creates an zigbee2mqtt/configuration.yaml file with only "homeassistant: true" in it. Even when I paste my old configuration in it and restart the add-on, the device list keeps empty.

Stooovie commented 2 years ago

See first point of https://github.com/zigbee2mqtt/hassio-zigbee2mqtt/releases/tag/v1.25.0-1, configuration can now be done via the frontend.

yes, me. I need to have the old config in the addon settings, otherwise it's zero devices for me.

Stooovie commented 2 years ago

with File Editor the file zigbee2mqtt/configuration.yaml

When I click reset, it creates an zigbee2mqtt/configuration.yaml file with only "homeassistant: true" in it. Even when I paste my old configuration in it and restart the add-on, the device list keeps empty.

same

Stooovie commented 2 years ago

When I do this, I lose all my connected devices. Does anyone has this problem too?

What exactly did you do ? I did not lose all my devices at all.

Did you delete the yaml file ? Don't do that at all, just only reset the config in Z2M

regards

I did that, and I lost all my devices. Pasted the old addon config back, restarted Z2M, devices are up again.

EDIT: somehow this procedure reset all my Z2M entity names to the device names, so I have to either restore from backup or redo all automations

mobiledude commented 2 years ago

Same here. tried several things. Only 2 devices recognized are hue motionsensors. the rest is not. What I also noticed is when re-applying the old config back in de zigbee2mqtt add-on. Entities are back in HA but allocated area's are removed. For now will go back to 1.24

malosaa commented 2 years ago

For this is like my conf looks like in the plugin after reset.

I have a ton of devices and had no issue at all, i even did test it on my test system and there its working aaswell

image

RUnning HASS OS on virtual box 7.5

thebatfink commented 2 years ago

Hi, for me, to solve this with an existing installation, in Add-ons / zigbee2mqtt / Configuration choose Reset configuration, check with File Editor the file zigbee2mqtt/configuration.yaml, if you see your previous options configuration, restart Host and warnings disappears

Thanks! This solved the problem for me, thanks. Was scratching my head with it.

mobiledude commented 2 years ago

Tried everything. Only two devices that show up are the 2 hue devices that are also available in devices.yaml. All other devices are lost.

This is my old config:


external_converters: []
devices: devices.yaml
groups: groups.yaml
homeassistant: true
permit_join: false
mqtt:
  base_topic: zigbee2mqtt
  server: mqtt://core-mosquitto:1883
  user: supersecretuser
  password: supersecretpassword
serial:
  port: /dev/ttyACM0
advanced:
  ikea_ota_use_test_url: true
  log_level: warn
  pan_id: 6754
  channel: 11
  network_key:
    - 1
    - 3
    - 5
    - 7
    - 9
    - 11
    - 13
    - 15
    - 0
    - 2
    - 4
    - 6
    - 8
    - 10
    - 12
    - 13
  availability_blocklist: []
  availability_passlist: []
device_options: {}
blocklist: []
passlist: []
queue: {}
frontend:
  port: 8099
experimental: {}
socat:
  enabled: false
  master: pty,raw,echo=0,link=/tmp/ttyZ2M,mode=777
  slave: tcp-listen:8485,keepalive,nodelay,reuseaddr,keepidle=1,keepintvl=1,keepcnt=5
  options: '-d -d'
  log: false```
dissyllabic commented 2 years ago

Hey everyone, I have been reading up on this topic, as I had the same issues that everybody else had: zigbee2Mqtt just didnt start. I solved the issue by installing SSL certificates (making my HA accessible via HTTPS) and by reading the logs in : ~\config\zigbee2mqtt\log and using them for troubleshooting. ALSO: this post, with the breaking change: https://github.com/zigbee2mqtt/hassio-zigbee2mqtt/releases/tag/v1.25.0-1 (from 21 days back) kind of got me troubleshooting on the right track (it says: As Zigbee2MQTT won't start with an MQTT server or serial adapter ) and my experience is that there seldom are "silver bullets" that would solve similar issues for everybody, instead, it almost always comes down to individual systems troubleshooting. So, good luck! BTW: I am running latest Homeassistant core on a Proxmox server (an 7 year old Dell Latitude laptop with an i7 and 16 GB RAM)

mobiledude commented 2 years ago

Actually the solution to my issues was quite easy. I thought I'd let you guys know.

steps: 1) upgrade to 1.25. 2) make sure everything works and your settings are automatically propogated to the config file in the zigbee directory 3) if so, create a backup from the yaml in the add-on just to be sure. 4) open supervisor log and look at your warnings. Then one by one delete the line in the add-on config related to the warning. 5) the add-on is restarted and the warning disappeared, check if everything still works. 6) Repeat untill all warnings are gone, and all your devices are still there.

No longer an issue here.

Stooovie commented 2 years ago

I did this. All devices gone. Pasting the addon config back and restarting the whole host resulted in all zigbee devices entity names being reset to the device names (for example, "binary_sensor.door" changed to "binary.sensor_xiaomi_mijia_door_sensor"), for whatever reason.

malosaa commented 2 years ago

I did this. All devices gone. Pasting the addon config back and restarting the whole host resulted in all zigbee devices entity names being reset to the device names (for example, "binary_sensor.door" changed to "binary.sensor_xiaomi_mijia_door_sensor"), for whatever reason.

Weird!

The only thing i did was that i pressed the wrong reset before.

this is the correct way image

and this is how it looks inside the folder. If you don't trust the upgrade just copy the database file, and all the yaml files and put them back after upgrade / reset

image

Beestkiller commented 2 years ago

I did this. All devices gone. Pasting the addon config back and restarting the whole host resulted in all zigbee devices entity names being reset to the device names (for example, "binary_sensor.door" changed to "binary.sensor_xiaomi_mijia_door_sensor"), for whatever reason.

Please check the data_path: in your config.

I lost all my devices aswel and saw that the data path is different, mine is data_path: /share/zigbee2mqtt/ and not data_path: /config/zigbee2mqtt

my config in the addon config page looks like this now and it works fine

data_path: /share/zigbee2mqtt
socat:
  enabled: false
  master: pty,raw,echo=0,link=/tmp/ttyZ2M,mode=777
  slave: tcp-listen:8485,keepalive,nodelay,reuseaddr,keepidle=1,keepintvl=1,keepcnt=5
  options: '-d -d'
  log: false
mqtt: {}
serial: {}
Stooovie commented 2 years ago

@Beestkiller thank you, that might be what caused my issue. Turns out I have BOTH configs existing for some reason (my config originated in 2019 so there's probably been some breaking changes that I didn't catch).

The addon config points to /share/zigbee2mqtt but both do physically exist and are fairly recently updated.

Which is the proper/default one?

wagaman commented 2 years ago

data_path: /config/zigbee2mqtt socat: enabled: false master: pty,raw,echo=0,link=/tmp/ttyZ2M,mode=777 slave: tcp-listen:8485,keepalive,nodelay,reuseaddr,keepidle=1,keepintvl=1,keepcnt=5 options: '-d -d' log: false mqtt: server: mqtt://core-mosquitto:1883 user: youruser password: yourpass serial: port: /dev/ttyUSB0 disable_led: false baudrate: 115200 rtscts: false advanced: adapter_concurrent: null transmit_power: 5

I am posting what works in the new version and be sure to use an USB extension with power when you use your adapter with Pie.

tjxism commented 2 years ago

After removing the redundant configurations on the add-on configuration page, I still the get the warning below: "WARNING (MainThread) [supervisor.addons.options] Unknown option 'base_topic' for Zigbee2mqtt (45df7312_zigbee2mqtt)". "ERROR (MainThread) [supervisor.api.ingress] Ingress error: Cannot connect to host 172.30.33.3:8099 ssl:default [Connect call failed ('172.30.33.3', 8099)]" Z2M seems to start fine and working

Now my configuration page only has data_path; socat; mqtt and serial:

Configuration Page now looks like this: data_path: /config/zigbee2mqtt socat: enabled: false master: pty,raw,echo=0,link=/tmp/ttyZ2M,mode=777 slave: tcp-listen:8485,keepalive,nodelay,reuseaddr,keepidle=1,keepintvl=1,keepcnt=5 options: '-d -d' log: false mqtt: base_topic: zigbee2mqtt user: {} password: {} serial: port: >- /dev/serial/by-id/usb-ITead_Sonoff_Zigbee_3.0_USB_DonglePlus{}

yaml looks like this: external_converters: [] devices:

smulle48 commented 2 years ago

After removing the redundant configurations on the add-on configuration page, I still the get the warning below:

"WARNING (MainThread) [supervisor.addons.options] Unknown option 'base_topic' for Zigbee2mqtt (45df7312_zigbee2mqtt)".

"ERROR (MainThread) [supervisor.api.ingress] Ingress error: Cannot connect to host 172.30.33.3:8099 ssl:default [Connect call failed ('172.30.33.3', 8099)]"

Z2M seems to start fine and working

Now my configuration page only has data_path; socat; mqtt and serial:

Configuration Page now looks like this:

data_path: /config/zigbee2mqtt

socat:

enabled: false

master: pty,raw,echo=0,link=/tmp/ttyZ2M,mode=777

slave: tcp-listen:8485,keepalive,nodelay,reuseaddr,keepidle=1,keepintvl=1,keepcnt=5

options: '-d -d'

log: false

mqtt:

base_topic: zigbee2mqtt

user: {}

password: {}

serial:

port: >-

/dev/serial/by-id/usb-ITead_Sonoff_Zigbee_3.0_USB_Dongle_Plus_{}

yaml looks like this:

external_converters: []

devices:

  • devices.yaml

groups:

  • groups.yaml

homeassistant: true

permit_join: false

mqtt:

user: {}

password: {}

server: mqtt://core-mosquitto:1883

serial:

port: >-

/dev/serial/by-id/usb-ITead_Sonoff_Zigbee_3.0_USB_Dongle_Plus_{}

advanced:

log_level: debug

pan_id: 6754

channel: 20

network_key:

- 1

- 3

- 5

- 7

- 9

- 11

- 13

- 15

- 0

- 2

- 4

- 6

- 8

- 10

- 12

- 13

availability_blocklist: []

availability_passlist: []

last_seen: ISO_8601_local

log_syslog:

app_name: Zigbee2MQTT

eol: /n

host: localhost

localhost: localhost

path: /dev/log

pid: process.pid

port: 123

protocol: tcp4

type: "5424"

transmit_power: 10

device_options: {}

blocklist: []

passlist: []

queue: {}

frontend:

port: 8099

experimental: {}

availability: true

Are you sure the port is correct? Can't remember any port with the {} at the end :-)

tjxism commented 2 years ago

Yeah… I just omitted that par in the {}. I get this warning but everything seems to work.

FSund commented 2 years ago

When I try removing the parts from the configuration I'm getting warnings about, I get an error when saving the new config. For example if I remove

mqtt:
  base_topic: zigbee2mqtt

I get the following error when I try saving

Failed to save add-on configuration, Missing option 'mqtt' in root in Zigbee2mqtt (45df7312_zigbee2mqtt). Got {'data_path': '/config/zigbee2mqtt', 'socat': {'enabled': False, 'master': 'pty,raw,echo=0,link=/tmp/ttyZ2M,mode=777', 'slave': 'tcp-listen:8485,keepalive,nodelay,reuseaddr,keepidle=1,keepintvl=1,keepcnt=5', 'options': '-d -d', 'log': False}, 'serial': {'port': '/dev/serial/by-id/usb-Silicon_Labs_slae.sh_cc2652rb_stick_-_slaesh_s_iot_stuff_00_12_4B_00_23_93_40_14-if00-port0'}, 'external_converters': [], 'devices': 'devices.yaml', 'groups': 'groups.yaml', 'homeassistant': True, 'permit_join': ...

image

github-actions[bot] commented 2 years ago

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days