haemishkyd commented 4 years ago

zigbee2mqtt add-on version (if edge, please report commit hash): 1.16.0

Operating environment (HassOS, Virtual Machine, Device/platform running Home Assistant): HassOS 4.15 Home Assistant 0.117.2 Mosquitto broker 5.1

Description of problem: I upgraded to 1.16.0 earlier and suddenly my system cannot connect to the MQTT broker. It now continues to give me. Zigbee2MQTT:error 2020-11-03 21:29:05: Not connected to MQTT server! I have looked through the change logs and nothing specific is mentioned about MQTT broker changes. The exact configuration given below was working perfectly on version 1.15.0. I have tried using core-mosquitto, localhost and the actual ip address with and without the port. None of this works. Has anything changed in the system to suggest what might be the issue here.

Your entire configuration from the frontend (with sensitive fields redacted):

data_path: /share/zigbee2mqtt
external_converters: []
devices: devices.yaml
groups: groups.yaml
homeassistant: true
permit_join: true
mqtt:
  base_topic: zigbee2mqtt
  server: 'mqtt://192.168.1.100:1833'
serial:
  port: /dev/ttyUSB0
advanced:
  log_level: info
  pan_id: 6754
  channel: 11
  network_key:
    - 1
    - 3
    - 5
    - 7
    - 9
    - 11
    - 13
    - 15
    - 0
    - 2
    - 4
    - 6
    - 8
    - 10
    - 12
    - 13
  availability_blocklist: []
  availability_passlist: []
device_options: {}
blocklist: []
passlist: []
queue: {}
frontend:
  port: 8099
experimental:
  new_api: true
socat:
  enabled: false
  master: 'pty,raw,echo=0,link=/dev/ttyZ2M,mode=777'
  slave: 'tcp-listen:8485,keepalive,nodelay,reuseaddr,keepidle=1,keepintvl=1,keepcnt=5'
  restartdelay: 1
  initialdelay: 1
  options: '-d -d'
  log: false

Your logs from Home Assistant

Zigbee2MQTT:error 2020-11-03 21:29:15: Not connected to MQTT server!
Zigbee2MQTT:error 2020-11-03 21:29:25: Not connected to MQTT server!
Zigbee2MQTT:error 2020-11-03 21:29:35: Not connected to MQTT server!
Zigbee2MQTT:error 2020-11-03 21:29:44: Not connected to MQTT server!
Zigbee2MQTT:error 2020-11-03 21:29:44: Cannot send message: topic: 'zigbee2mqtt/zigbee_button_1', payload: '{"battery":100,"linkquality":60,"voltage":3100}
Zigbee2MQTT:error 2020-11-03 21:29:45: Not connected to MQTT server!
Zigbee2MQTT:error 2020-11-03 21:29:55: Not connected to MQTT server!

ciotlosm commented 4 years ago

You can try using https://github.com/zigbee2mqtt/hassio-zigbee2mqtt as repo and install the add-on from there and just keep:

mqtt:
  base_topic: zigbee2mqtt

And allow the add-on to auto-discover mqtt. See if that works.

rosaces commented 4 years ago

I've got the same issue, since the update. Do I really need to reinstall the whole thing? Can I roll back to 1.15.0?

ciotlosm commented 4 years ago

Rollback to 1.15.0 is not easy. You would need to fork the repo and update some versions, but "reinstall" is not really that costly. As long as you backup your config, rest of settings should be on the storage in zigbee2mqtt folder and should work.

I really recommended to try https://github.com/zigbee2mqtt/hassio-zigbee2mqtt and install 1.16.0 from there because it contains mqtt auto discovery with the settings mentioned above.

jackwilsdon commented 4 years ago

I did some digging on this, and a temporary fix seems to be to enable authentication on the MQTT server (you can leave anonymous enabled too) and to use a username/password on zigbee2mqtt.

It turns out that this is a bug introduced in the new version of MQTT.js which zigbee2mqtt updated to in 1.16.0, and it only exhibits itself when connecting to the Mosquitto Home Assistant Add-on (something to do with the auth plugin it uses it seems). I've opened https://github.com/mqttjs/MQTT.js/pull/1197 to fix the issue in the library, which zigbee2mqtt can use once it has been merged.

haemishkyd commented 4 years ago

Thanks for your feedback guys. @jackwilsdon I have tried to enabling authentication on mosquitto and putting those credentials into zigbee2mqtt. This did not change anything. There was still no connection to the MQTT server. I am not sure if I did that correctly or not. In Mosquitto config I added a username/password to the logins section and put those same credentials into zigbee2mqtt. I assume this was all I needed to do. If so this did not work.

@ciotlosm I have also swapped now to https://github.com/zigbee2mqtt/hassio-zigbee2mqtt. In this, as suggested, I have let the auto discover run. This seemed to connect to the mqtt server. However when I push a zigbee button the data comes up in the log. It says the button has been pushed:

Zigbee2MQTT:debug 2020-11-04 17:57:17: Received Zigbee message from 'zigbee_button_1', type 'commandToggle', cluster 'genOnOff', data '{}' from endpoint 1 with groupID 0
Zigbee2MQTT:info  2020-11-04 17:57:17: MQTT publish: topic 'zigbee2mqtt/zigbee_button_1', payload '{"action":"single","battery":100,"linkquality":44,"voltage":4000}'
Zigbee2MQTT:info  2020-11-04 17:57:17: MQTT publish: topic 'zigbee2mqtt/zigbee_button_1', payload '{"action":"","battery":100,"linkquality":44,"voltage":4000}'
Zigbee2MQTT:info  2020-11-04 17:57:17: MQTT publish: topic 'zigbee2mqtt/zigbee_button_1/action', payload 'single'

I am running MQTT explorer connected to the same broker and I get nothing. Needless to say nothing else works either. I am out of ideas so if anyone else has anything to suggest I would appreciate it.

ciotlosm commented 4 years ago

@haemishkyd if you see the logs for mqtt, it means stuff is working. Are you sure you are looking at the right topic with MQTT explorer?

UnderpantsGnome commented 4 years ago

@haemishkyd if you see the logs for mqtt, it means stuff is working. Are you sure you are looking at the right topic with MQTT explorer?

@ciotlosm I'm seeing the same behavior.

Log output from Supervisor -> z2m -> Log

Zigbee2MQTT:info  2020-11-04 14:52:13: MQTT publish: topic 'zigbee2mqtt/living-room-light', payload '{"consumption":1.93,"energy":1.93,"linkquality":105,"power":0,"state":"OFF","temperature":30}'
Zigbee2MQTT:info  2020-11-04 14:54:31: MQTT publish: topic 'zigbee2mqtt/bonus-room', payload '{"battery":100,"humidity":55.91,"linkquality":54,"pressure":993.3,"temperature":21.26,"voltage":3005}'

In another tab running mosquitto_sub -h 10.0.0.34 -v -t "#" in the Terminal shows no output.

z2m MQTT config:

  base_topic: zigbee2mqtt
  server: 'mqtt://10.0.0.34:1883'
  user: user
  password: passwd

bigboban commented 4 years ago

metoo

Fixed by downgrade to version 15.

UnderpantsGnome commented 4 years ago

@bigboban what steps did you have to take to downgrade though the add ons?

ciotlosm commented 4 years ago

Tagging @Koenkk as this might be related to libraries in zigbee2mqtt

jackwilsdon commented 4 years ago

If it helps at all, I bisected this down to https://github.com/Koenkk/zigbee2mqtt/pull/4830 in the end (mqtt 4.2.2 introduced the issue).

https://github.com/mqttjs/MQTT.js/pull/1197 has an explanation of the issue I was seeing.

bigboban commented 4 years ago

@UnderpantsGnome Downgraded using selective restore from snapshot - only this plugin.

Koenkk commented 4 years ago

@jackwilsdon big thanks for your investigation here, can this be fixed by downgrading mqtt 4.2.1? If yes I will push an update today which will be available in the edge version.

jackwilsdon commented 4 years ago

@Koenkk downgrading to 4.2.1 with npm install --save mqtt@4.2.1 on the dev branch seems to fix it for me :tada:

$ git checkout dev
$ npm ci
$ npm start
...
Zigbee2MQTT:error 2020-11-05 12:22:12: Not connected to MQTT server!
$ npm install --save mqtt@4.2.1
$ npm start
...
Zigbee2MQTT:info  2020-11-05 12:23:15: Connected to MQTT server

It might be best pinning it to 4.2.1 in package.json too just so that the update bot doesn't come round again 😆

Koenkk commented 3 years ago

@jackwilsdon great, I've released a new Zigbee2MQTT version: 1.16.1 @ciotlosm can you trigger a new release?

rickkdotnet commented 3 years ago

I can confirm that 1.16.1 fixes the issue for me, thanks!

ciotlosm commented 3 years ago

@Koenkk new build is finished and published.

@haemishkyd can you confirm new version fixes your issue?

@Koenkk off topic: How can I get edge pipeline to trigger on the new repo in sync with your dev branch? Are you running them yourself at the moment?

https://github.com/zigbee2mqtt/hassio-zigbee2mqtt https://dev.azure.com/zigbee2mqtt/Zigbee2mqtt%20Add-on/_build?definitionId=9 https://github.com/zigbee2mqtt/hassio-zigbee2mqtt/blob/dev/pipeline/edge-build-pipeline.yml

I'm preparing for doing a switch in the near future to the repo and that is one of the features missing. Currently build times in the new repo are faster, and the workflow is fixed (publish of version can be done after build is finished). Screenshot 2020-11-05 at 19 54 48

Screenshot 2020-11-05 at 19 48 31

As it can be seen the build times are vastly improved current	new
27m 55s	13m 17s

Also a few other improvements visible in changelog like mqtt auto discovery.

UnderpantsGnome commented 3 years ago

1.16.1 resolves the auth issue, but I still don't get data published to the MQTT broker, I can see things updating on the z2m frontend. 🤷‍♂️

Koenkk commented 3 years ago

@ciotlosm it's triggered here: https://github.com/Koenkk/zigbee2mqtt/blob/6b32f30654831cd94fc55fa813f8674d6ef09124/.github/workflows/ci.yml#L79

ciotlosm commented 3 years ago

Can you add one for https://dev.azure.com/zigbee2mqtt/Zigbee2mqtt%20Add-on/_build?definitionId=9 as well please?

jackwilsdon commented 3 years ago

@UnderpantsGnome do you have any ACLs configured? Try connecting to the MQTT sever with another client using the credentials you're using with zigbee2mqtt and see if messages published by that client can be seen by others.

UnderpantsGnome commented 3 years ago

@jackwilsdon no ACL setup, anonymous logins, I can publish fine from MQTT.fx, nothing else has changed other than the upgrade to 1.16 yesterday.

z2m config.

data_path: /share/zigbee2mqtt
external_converters: []
devices: devices.yaml
groups: groups.yaml
homeassistant: true
permit_join: true
mqtt:
  base_topic: zigbee2mqtt
  server: 'mqtt://10.0.0.34'
serial:
  port: /dev/ttyUSB0
advanced:
  log_level: info
  pan_id: 6754
  channel: 11
  network_key:
    - 1
    - 3
    - 5
    - 7
    - 9
    - 11
    - 13
    - 15
    - 0
    - 2
    - 4
    - 6
    - 8
    - 10
    - 12
    - 13
  availability_blocklist: []
  availability_passlist: []
device_options: {}
blocklist: []
passlist: []
queue: {}
frontend:
  port: 8099
experimental:
  new_api: true
socat:
  enabled: false
  master: 'pty,raw,echo=0,link=/dev/ttyZ2M,mode=777'
  slave: 'tcp-listen:8485,keepalive,nodelay,reuseaddr,keepidle=1,keepintvl=1,keepcnt=5'
  restartdelay: 1
  initialdelay: 1
  options: '-d -d'
  log: false

broker config

logins: []
anonymous: true
customize:
  active: true
  folder: mosquitto
certfile: fullchain.pem
keyfile: privkey.pem
require_certificate: false

jackwilsdon commented 3 years ago

We might be best taking this to the zigbee2mqtt discourse - it's a bit easier to discuss it there.

ciotlosm commented 3 years ago

I'll close the item as the auth is fixed.

kevincaradant commented 3 years ago

Hi

I just upgraded it via docker on 1.16.1 Zigbee2MQTT and after that all my devices goes unvailable on HA :(

I didn't understand if it's an issue about zigbee2mqtt or HA addon MQTT ? I'm using Home Assistant with Docker and Zigbee2MQTT with Docker.

@UnderpantsGnome Do you have created an issue for this ? Somewhere ? I'm a little bit confused about this issue.

I force for the moment the v1.15.0 of Zigbee2MQTT with docker to reload the old version and everything works again.

Regards

UnderpantsGnome commented 3 years ago

@kevincaradant I completely removed the addon, and the repo for it, then added them back in (to be sure I was using the correct repo) and things started working again.

kevincaradant commented 3 years ago

I tried twice your solution but that doesn't seems works well :/ When you said repo, what do you mean, because I was just in "integration" section. I searched "MQTT" plugin and it was already in it.

Indisponible = unavailable

So something wrong

I got this error for each device connected:

 [homeassistant.util.logging] Exception in async_discover when dispatching 'mqtt_discovery_new_light_mqtt': ({'availability': [{'topic': 'zigbee2mqtt/bridge/state'}, {'topic': 'zigbee2mqtt/gledopto_1_bathroom/availability'}], 'brightness': True, 'brightness_scale': 254, 'color_temp': True, 'command_topic': 'zigbee2mqtt/gledopto_1_bathroom/set', 'device': {'identifiers': ['zigbee2mqtt_0x00124b001ed01121'], 'manufacturer': 'Gledopto', 'model': 'Smart 4W E14 RGB / CCT LED bulb (GL-B-001ZS)', 'name': 'gledopto_1_bathroom', 'sw_version': 'Zigbee2MQTT 1.16.1'}, 'effect': True, 'effect_list': ['blink', 'breathe', 'okay', 'channel_change', 'finish_effect', 'stop_effect'], 'hs': False, 'json_attributes_topic': 'zigbee2mqtt/gledopto_1_bathroom', 'name': 'gledopto_1_bathroom_light', 'schema': 'json', 'state_topic': 'zigbee2mqtt/gledopto_1_bathroom', 'unique_id': '0x00124b001ed01121_light_zigbee2mqtt', 'xy': True, 'platform': 'mqtt'},)
home-assistant   | Traceback (most recent call last):
home-assistant   |   File "/usr/src/homeassistant/homeassistant/components/mqtt/light/__init__.py", line 52, in async_discover
home-assistant   |     config = PLATFORM_SCHEMA(discovery_payload)
home-assistant   |   File "/usr/local/lib/python3.7/site-packages/voluptuous/validators.py", line 208, in __call__
home-assistant   |     return self._exec((Schema(val) for val in self.validators), v)
home-assistant   |   File "/usr/local/lib/python3.7/site-packages/voluptuous/validators.py", line 287, in _exec
home-assistant   |     raise e if self.msg is None else AllInvalid(self.msg, path=path)
home-assistant   |   File "/usr/local/lib/python3.7/site-packages/voluptuous/validators.py", line 283, in _exec
home-assistant   |     v = func(v)
home-assistant   |   File "/usr/local/lib/python3.7/site-packages/voluptuous/schema_builder.py", line 272, in __call__
home-assistant   |     return self._compiled([], data)
home-assistant   |   File "/usr/local/lib/python3.7/site-packages/voluptuous/schema_builder.py", line 817, in validate_callable
home-assistant   |     return schema(data)
home-assistant   |   File "/usr/src/homeassistant/homeassistant/components/mqtt/light/__init__.py", line 30, in validate_mqtt_light
home-assistant   |     return schemas[value[CONF_SCHEMA]](value)
home-assistant   |   File "/usr/local/lib/python3.7/site-packages/voluptuous/schema_builder.py", line 272, in __call__
home-assistant   |     return self._compiled([], data)
home-assistant   |   File "/usr/local/lib/python3.7/site-packages/voluptuous/schema_builder.py", line 594, in validate_dict
home-assistant   |     return base_validate(path, iteritems(data), out)
home-assistant   |   File "/usr/local/lib/python3.7/site-packages/voluptuous/schema_builder.py", line 432, in validate_mapping
home-assistant   |     raise er.MultipleInvalid(errors)
home-assistant   | voluptuous.error.MultipleInvalid: extra keys not allowed @ data['availability']

ciotlosm commented 3 years ago

For those that are still having problems getting their device data, I was able to reproduce one of the scenarios.

The steps to reproduce would be:

Upgrade zigbee2mqtt
Upgrade home assistant after

The issue is that when upgrading zigbee2mqtt, on restart it will send "offline" state for all devices when shutting down. This is sent with a retain flag.

When home assistant starts, it will see the retain flag, and start with all devices unavailable for zigbee2mqtt integration. Even if zigbee2mqtt is connected to home assistant via mqtt and messages are working, your devices will not update.

To fix this you must restart zigbee2mqtt once to make sure that the online message is sent for device availability, as this is done at startup.

You might want to look at https://www.zigbee2mqtt.io/information/availability.html and try some settings to fix this.

For those that are still having problems, please open a new issue with the right details.

kevincaradant commented 3 years ago

Thanks you @ciotlosm

I was not using the latest version of Home Assistant because I got some issues with it until the last month but I retried today to update HA and everything works as expected 👍

Then I upgraded Zigbee2MQTT in 1.16.1 and I restarted it.

HA is still working as expected so that's good. Pb solved

haemishkyd commented 3 years ago

Hi,

I have followed all the advice given here (I think) and still nothing will publish to the MQTT broker. I am now on Zigbee2MQTT :1.16.1 Mosquitto broker: 5.1 HassOS : 4.15 Home Assistant :0.117.5

I have confirmed that from another client I can publish through the broker.
I have unistalled the add on - removed the repository and added it again and then re-installed the add-on
I have tried adding a user on mosquitto and then putting those credentials into zigbee2mqtt
I have tried without the server (auto descovery of broker) and with the server in the config.
I can publish to a topic from within HomeAssistant manually

My Zigbee2MQTT config:

data_path: /config/zigbee2mqtt
external_converters: []
devices: devices.yaml
groups: groups.yaml
homeassistant: true
permit_join: false
mqtt:
  base_topic: zigbee2mqtt
  server: 'mqtt://core-mosquitto:1883'
  user: haemish
  password: yourpassword
serial:
  port: /dev/ttyUSB0
advanced:
  log_level: debug
  pan_id: 6754
  channel: 11
  network_key:
    - 1
    - 3
    - 5
    - 7
    - 9
    - 11
    - 13
    - 15
    - 0
    - 2
    - 4
    - 6
    - 8
    - 10
    - 12
    - 13
  availability_blocklist: []
  availability_passlist: []
device_options: {}
blocklist: []
passlist: []
queue: {}
frontend:
  port: 8099
experimental:
  new_api: true
socat:
  enabled: false
  master: 'pty,raw,echo=0,link=/dev/ttyZ2M,mode=777'
  slave: 'tcp-listen:8485,keepalive,nodelay,reuseaddr,keepidle=1,keepintvl=1,keepcnt=5'
  options: '-d -d'
  log: false

My mosquitto config:

logins:
  - username: haemish
    password: yourpassword
anonymous: true
customize:
  active: true
  folder: mosquitto
certfile: fullchain.pem
keyfile: privkey.pem
require_certificate: false

I have tried so many permutations of this now but still I can't seem to get this to work. Any thoughts or help would be appreciated.

jackwilsdon commented 3 years ago

@haemishkyd What's in your mosquitto customize folder? (I notice you have it enabled) Are there any ACLs in there?

ciotlosm commented 3 years ago

@haemishkyd I'd recommend you try the version with auto-discovery and allow it to fetch the users for you (https://github.com/zigbee2mqtt/hassio-zigbee2mqtt) and see how that works. Also please post logs when opening an issue.

haemishkyd commented 3 years ago

Hello,

Thanks for your responses.

@ciotlosm I am using the version with auto-discovery. My apologies. I should have added logs. Please see the logs after a restart:

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] socat.sh: executing... 
[19:05:49] INFO: Socat not enabled, marking service as down
[cont-init.d] socat.sh: exited 0.
[cont-init.d] zigbee2mqtt.sh: executing... 
[19:05:51] INFO: MQTT service found, fetching server detail ...
[19:05:52] INFO: MQTT credentials not configured, trying to auto-discovering ...
[19:05:53] INFO: Configuring'addons' mqtt user
[19:05:53] INFO: Previous config file found, checking backup
[19:05:53] INFO: Creating backup config in '/config/zigbee2mqtt/.configuration.yaml.bk'
[19:05:53] INFO: Adjusting Zigbee2mqtt core yaml config with add-on quirks ...
[cont-init.d] zigbee2mqtt.sh: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[19:05:55] INFO: Handing over control to Zigbee2mqtt Core ...
> zigbee2mqtt@1.16.1 start /app
> node index.js
Zigbee2MQTT:debug 2020-11-09 21:06:02: Loaded state from file /config/zigbee2mqtt/state.json
Zigbee2MQTT:info  2020-11-09 21:06:02: Logging to console and directory: '/config/zigbee2mqtt/log/2020-11-09.21-06-00' filename: log.txt
Zigbee2MQTT:info  2020-11-09 21:06:02: Starting Zigbee2MQTT version 1.16.1 (commit #1.16.1)
Zigbee2MQTT:info  2020-11-09 21:06:02: Starting zigbee-herdsman...
Zigbee2MQTT:debug 2020-11-09 21:06:02: Using zigbee-herdsman with settings: '{"adapter":{"concurrent":null},"backupPath":"/config/zigbee2mqtt/coordinator_backup.json","databaseBackupPath":"/config/zigbee2mqtt/database.db.backup","databasePath":"/config/zigbee2mqtt/database.db","network":{"channelList":[11],"extendedPanID":[221,221,221,221,221,221,221,221],"networkKey":"HIDDEN","panID":6754},"serialPort":{"path":"/dev/ttyUSB0"}}'
Zigbee2MQTT:info  2020-11-09 21:06:05: zigbee-herdsman started
Zigbee2MQTT:info  2020-11-09 21:06:05: Coordinator firmware version: '{"meta":{"maintrel":3,"majorrel":2,"minorrel":6,"product":0,"revision":20190608,"transportrev":2},"type":"zStack12"}'
Zigbee2MQTT:debug 2020-11-09 21:06:05: Zigbee network parameters: {"channel":11,"extendedPanID":"0xdddddddddddddddd","panID":6754}
Zigbee2MQTT:info  2020-11-09 21:06:05: Currently 1 devices are joined:
Zigbee2MQTT:info  2020-11-09 21:06:05: zigbee_button_1 (0x00124b001cd61073): SNZB-01 - SONOFF Wireless button (EndDevice)
Zigbee2MQTT:info  2020-11-09 21:06:05: Zigbee: disabling joining new devices.
Zigbee2MQTT:info  2020-11-09 21:06:05: Started frontend on port 8099
Zigbee2MQTT:info  2020-11-09 21:06:05: Connecting to MQTT server at mqtt://core-mosquitto:1883
Zigbee2MQTT:info  2020-11-09 21:06:06: Connected to MQTT server
Zigbee2MQTT:info  2020-11-09 21:06:06: MQTT publish: topic 'zigbee2mqtt/bridge/state', payload 'online'
Zigbee2MQTT:info  2020-11-09 21:06:06: MQTT publish: topic 'zigbee2mqtt/zigbee_button_1', payload '{"battery":100,"voltage":3100}'
Zigbee2MQTT:info  2020-11-09 21:06:06: MQTT publish: topic 'zigbee2mqtt/bridge/info', payload '{"commit":"1.16.1","config":{"advanced":{"adapter_concurrent":null,"availability_blacklist":[],"availability_blocklist":[],"availability_passlist":[],"availability_timeout":0,"availability_whitelist":[],"cache_state":true,"cache_state_persistent":true,"cache_state_send_on_startup":true,"channel":11,"elapsed":false,"ext_pan_id":[221,221,221,221,221,221,221,221],"homeassistant_discovery_topic":"homeassistant","homeassistant_legacy_triggers":true,"homeassistant_status_topic":"hass/status","last_seen":"disable","legacy_api":true,"log_directory":"/config/zigbee2mqtt/log/%TIMESTAMP%","log_file":"log.txt","log_level":"debug","log_output":["console","file"],"log_rotation":true,"log_syslog":{},"pan_id":6754,"report":false,"soft_reset_timeout":0,"timestamp_format":"YYYY-MM-DD HH:mm:ss"},"ban":[],"blocklist":[],"device_options":{},"devices":{"0x00124b001cd61073":{"friendly_name":"zigbee_button_1"}},"experimental":{"new_api":true,"output":"json"},"external_converters":[],"frontend":{"port":8099},"groups":{},"homeassistant":true,"map_options":{"graphviz":{"colors":{"fill":{"coordinator":"#e04e5d","enddevice":"#fff8ce","router":"#4ea3e0"},"font":{"coordinator":"#ffffff","enddevice":"#000000","router":"#ffffff"},"line":{"active":"#009900","inactive":"#994444"}}}},"mqtt":{"base_topic":"zigbee2mqtt","include_device_information":false,"server":"mqtt://core-mosquitto:1883","user":"addons"},"passlist":[],"permit_join":false,"queue":{},"serial":{"disable_led":false,"port":"/dev/ttyUSB0"},"whitelist":[]},"coordinator":{"meta":{"maintrel":3,"majorrel":2,"minorrel":6,"product":0,"revision":20190608,"transportrev":2},"type":"zStack12"},"log_level":"debug","network":{"channel":11,"extended_pan_id":"0xdddddddddddddddd","pan_id":6754},"permit_join":false,"version":"1.16.1"}'
Zigbee2MQTT:info  2020-11-09 21:06:06: MQTT publish: topic 'zigbee2mqtt/bridge/devices', payload '[{"definition":null,"endpoints":{"1":{"bindings":[],"clusters":{"input":[],"output":[]}},"11":{"bindings":[],"clusters":{"input":["ssIasAce"],"output":["ssIasZone","ssIasWd"]}},"110":{"bindings":[],"clusters":{"input":[],"output":[]}},"12":{"bindings":[],"clusters":{"input":[],"output":[]}},"13":{"bindings":[],"clusters":{"input":["genOta"],"output":[]}},"2":{"bindings":[],"clusters":{"input":[],"output":[]}},"242":{"bindings":[],"clusters":{"input":[],"output":[]}},"3":{"bindings":[],"clusters":{"input":[],"output":[]}},"4":{"bindings":[],"clusters":{"input":[],"output":[]}},"47":{"bindings":[],"clusters":{"input":[],"output":[]}},"5":{"bindings":[],"clusters":{"input":[],"output":[]}},"6":{"bindings":[],"clusters":{"input":[],"output":[]}},"8":{"bindings":[],"clusters":{"input":[],"output":[]}}},"friendly_name":"Coordinator","ieee_address":"0x00124b001fc93c57","interview_completed":true,"interviewing":false,"network_address":0,"supported":false,"type":"Coordinator"},{"date_code":"20191108","definition":{"description":"Wireless button","exposes":[{"access":"r","name":"battery","property":"battery","type":"numeric","unit":"%"},{"access":"r","name":"action","property":"action","type":"enum","values":["single","double","long"]},{"access":"r","name":"linkquality","property":"linkquality","type":"numeric","unit":"lqi"}],"model":"SNZB-01","vendor":"SONOFF"},"endpoints":{"1":{"bindings":[{"cluster":"genOnOff","target":{"endpoint":1,"ieee_address":"0x00124b001fc93c57","type":"endpoint"}},{"cluster":"genPowerCfg","target":{"endpoint":1,"ieee_address":"0x00124b001fc93c57","type":"endpoint"}}],"clusters":{"input":["genBasic","genIdentify","genPowerCfg"],"output":["genOnOff","genIdentify"]}}},"friendly_name":"zigbee_button_1","ieee_address":"0x00124b001cd61073","interview_completed":true,"interviewing":false,"network_address":44513,"power_source":"Battery","supported":true,"type":"EndDevice"}]'
Zigbee2MQTT:info  2020-11-09 21:06:06: MQTT publish: topic 'zigbee2mqtt/bridge/groups', payload '[]'
Zigbee2MQTT:info  2020-11-09 21:06:07: MQTT publish: topic 'zigbee2mqtt/bridge/config', payload '{"commit":"1.16.1","coordinator":{"meta":{"maintrel":3,"majorrel":2,"minorrel":6,"product":0,"revision":20190608,"transportrev":2},"type":"zStack12"},"log_level":"debug","network":{"channel":11,"extendedPanID":"0xdddddddddddddddd","panID":6754},"permit_join":false,"version":"1.16.1"}'
Zigbee2MQTT:info  2020-11-09 21:06:07: MQTT publish: topic 'homeassistant/sensor/0x00124b001cd61073/battery/config', payload '{"availability":[{"topic":"zigbee2mqtt/bridge/state"},{"topic":"zigbee2mqtt/zigbee_button_1/availability"}],"device":{"identifiers":["zigbee2mqtt_0x00124b001cd61073"],"manufacturer":"SONOFF","model":"Wireless button (SNZB-01)","name":"zigbee_button_1","sw_version":"Zigbee2MQTT 1.16.1"},"device_class":"battery","json_attributes_topic":"zigbee2mqtt/zigbee_button_1","name":"zigbee_button_1_battery","state_topic":"zigbee2mqtt/zigbee_button_1","unique_id":"0x00124b001cd61073_battery_zigbee2mqtt","unit_of_measurement":"%","value_template":"{{ value_json.battery }}"}'
Zigbee2MQTT:info  2020-11-09 21:06:07: MQTT publish: topic 'homeassistant/sensor/0x00124b001cd61073/action/config', payload '{"availability":[{"topic":"zigbee2mqtt/bridge/state"},{"topic":"zigbee2mqtt/zigbee_button_1/availability"}],"device":{"identifiers":["zigbee2mqtt_0x00124b001cd61073"],"manufacturer":"SONOFF","model":"Wireless button (SNZB-01)","name":"zigbee_button_1","sw_version":"Zigbee2MQTT 1.16.1"},"icon":"mdi:gesture-double-tap","json_attributes_topic":"zigbee2mqtt/zigbee_button_1","name":"zigbee_button_1_action","state_topic":"zigbee2mqtt/zigbee_button_1","unique_id":"0x00124b001cd61073_action_zigbee2mqtt","value_template":"{{ value_json.action }}"}'
Zigbee2MQTT:info  2020-11-09 21:06:07: MQTT publish: topic 'homeassistant/sensor/0x00124b001cd61073/linkquality/config', payload '{"availability":[{"topic":"zigbee2mqtt/bridge/state"},{"topic":"zigbee2mqtt/zigbee_button_1/availability"}],"device":{"identifiers":["zigbee2mqtt_0x00124b001cd61073"],"manufacturer":"SONOFF","model":"Wireless button (SNZB-01)","name":"zigbee_button_1","sw_version":"Zigbee2MQTT 1.16.1"},"icon":"mdi:signal","json_attributes_topic":"zigbee2mqtt/zigbee_button_1","name":"zigbee_button_1_linkquality","state_topic":"zigbee2mqtt/zigbee_button_1","unique_id":"0x00124b001cd61073_linkquality_zigbee2mqtt","unit_of_measurement":"lqi","value_template":"{{ value_json.linkquality }}"}'
Zigbee2MQTT:debug 2020-11-09 21:07:27: Received Zigbee message from 'zigbee_button_1', type 'commandToggle', cluster 'genOnOff', data '{}' from endpoint 1 with groupID 0
Zigbee2MQTT:info  2020-11-09 21:07:27: MQTT publish: topic 'zigbee2mqtt/zigbee_button_1', payload '{"action":"single","battery":100,"linkquality":49,"voltage":3100}'
Zigbee2MQTT:info  2020-11-09 21:07:27: MQTT publish: topic 'zigbee2mqtt/zigbee_button_1', payload '{"action":"","battery":100,"linkquality":49,"voltage":3100}'
Zigbee2MQTT:info  2020-11-09 21:07:27: MQTT publish: topic 'homeassistant/device_automation/0x00124b001cd61073/action_single/config', payload '{"automation_type":"trigger","device":{"identifiers":["zigbee2mqtt_0x00124b001cd61073"],"manufacturer":"SONOFF","model":"Wireless button (SNZB-01)","name":"zigbee_button_1","sw_version":"Zigbee2MQTT 1.16.1"},"payload":"single","subtype":"single","topic":"zigbee2mqtt/zigbee_button_1/action","type":"action"}'
Zigbee2MQTT:info  2020-11-09 21:07:27: MQTT publish: topic 'zigbee2mqtt/zigbee_button_1/action', payload 'single'

You will see at the end of the log I pushed a button. This publication is not seen on any MQTT sniffer. I am using two different ones on my network. If I publish from any other client I see the message in these MQTT sniffers.

@jackwilsdon my accesscontrolist file contains:

topic readwrite #

and acl.conf file:

acl_file /share/mosquitto/accesscontrollist

jackwilsdon commented 3 years ago

topic readwrite # only allows read and write for the anonymous user. To enable read and write for other users, you need to add the following to the end of your config (after your existing topic readwrite #).

user haemish
topic readwrite #

Reference for acl file contents: https://mosquitto.org/man/mosquitto-conf-5.html#idm45473494697792

For 1.16.1 you shouldn't need to use authentication anyway now, so feel free to remove it again.

haemishkyd commented 3 years ago

Thanks for the advice I will try that. Does that means that zigbee2mqtt does NOT work with anonymous users? I have noticed that if the user is not set the config file seems to default to the "addon" user in home assistant. Would I be able to just add the "addon" user to my acl file?

jackwilsdon commented 3 years ago

There was an issue in zigbee2mqtt 1.16.0 where anonymous users no longer worked. This has been fixed in 1.16.1. I'm not sure how the addon user works, as I've not used the autodiscovery version of the addon. You should be able to add whatever the addon user's name is to your acl file as I listed above though.

haemishkyd commented 3 years ago

Thanks for the advice. It seems to have been an issue with the ACLs as suggested.

oz9els commented 3 years ago

I experienced high CPU (100%) after upgrading to 5.1.1 (from the Official add-ons) I could not pinpoint the issue to any specific component. I stopped the zigbee2mqtt and OpenZwave - This got the CPU down to the usual 0.2% I have a integration with my floor-heating, also talking MQTT (with normal user/password), it actually worked. (but I did'nt actually take note of that) At first, I did not associate my problem this with the system-Users (addons & homeassistant) issue - But i'm sure it actually was the root cause of my problems. I re-installed the "Mosquitto broker" no luck, Rolled back to version 5.1, no Luck. Changed user/password, No Luck. Cleaned up the config etc, no luck. After reading this thread, I got the hint I was missing.. Looking closer to the log, I found some unauthorized users. After a re-install of the integration, everything start looking MUCH MUCH better. Not sure what caused the out-of-sync passwords, but I think that the problem...

Once in a while I get some: "Socket error on client , disconnecting." from my OpenZwave add-on.

Only problem now: Do I dare to upgrade to 5.1.1 ;-) - I will probably wait some weeks..

danielwelch / hassio-zigbee2mqtt

Zigbee2mqtt Cannot Connect to MQTT server #460

metoo