[Feature request]: Improve logging in Z2M

[martyzeq]

Is your feature request related to a problem? Please describe

after this fix is logging useless for me - not enough in Info loglevel, too much in Debug loglevel

Describe the solution you'd like

Is please possible to extend loglevels like in syslog etc for example in range 0-7 to cover big gap between Info and Debug ? I personally use remote syslog, but if someone is using microSD, eMMC etc, debug would kill it soon and info doesn't tell much, although it should be second most talkative loglevel. Thank you

Describe alternatives you've considered

or there might be customisation, what messages user wants to see in z2m log in form of checklist

Additional context

[johnholman]

Agree there is a big gap between info and debug. I miss seeing the MQTT publish messages at info level, and debug is far too much for routine troubleshooting. Perhaps use notice for what is now info, and add MQTT publish messages back to info?

[SargonofAssyria]

I agree, MQTT publish messages are missing for me too.

[Nerivec]

MQTT Publish lines are far too verbose for info, especially for larger networks, or people with high-update-rate devices (like power monitoring). But a new level just for that line would be overkill. Though a new level could be introduced after more feedback is gathered on the logging unification, if more lines need a new level. In the meantime, check the preview of the docs for next release on new logging settings: https://github.com/Koenkk/zigbee2mqtt.io/pull/2737/files You should be able to reduce debug level to whatever you want by filtering the namespace.

[maxifly]

I agree, In addition, I do not see debug level messages in the web interface. Which is also inconvenient for me.

[maxifly]

You should be able to reduce debug level to whatever you want by filtering the namespace.

Why not then add a setting that allows these messages to be displayed at the info level? Especially considering that debug messages are no longer displayed in the web interface?

[vsigler]

I just came here to file this very same ticket. I just did a fresh installation and the debug logging looks like a mess.

I am also mainly after the MQTT publish logs. I looked at the new logging settings preview, @Nerivec, but I get the feeling it is trying to reinvent the wheel or maybe a square wheel. I haven't seen the code of the logging subsystem, but could we just use a common approach to configure the logger hierarchy? I already see the structure in the logs. So how about something like:

advanced:
  log_level: info # base/"root" log level
  logger_levels: # optional fine-tuning of specific loggers
    zh_controller: debug
    zh_controller_endpoint: info
    z2m_mqtt_publish: debug 
    zh_zstack: warn

Delimiters can be different, it can even be a tree structure, but I just went with the simplest config to illustrate the approach. I'd say just about any decent logging framework allows to tune levels for different areas of the application. Such a setup is more powerful and possibly extendable compared to adding a regular expression to suppress certain logs. Ideally I do not care what namespaces are there. I'd like to just modify setting for a particular one I am interested in and keep the rest at what they are by default.

[DumahBrazorf]

I was just asking myself "where the heck are mqtt messages logs?", but i don't get anything even in debug logging level.

[LaurentChardin]

I agree, In addition, I do not see debug level messages in the web interface. Which is also inconvenient for me.

https://www.zigbee2mqtt.io/guide/usage/debug.html#publishing-to-mqtt-and-to-frontend : you can actually activate the debug logs in the UI with this setting

And then you can play with the regex to filter out the debug logs, using the namespaces.

[DumahBrazorf]

advanced: log_debug_to_mqtt_frontend: true

there should be a checkbox option in the frontend for this.

[LaurentChardin]

advanced: log_debug_to_mqtt_frontend: true

there should be a checkbox option in the frontend for this.

There is:

[johnholman]

Just tried to use this to show only z2m: MQTT messages in the debug log. This involves a double negative, i.e. ignore everything except messages beginning z2m: MQTT, which is a bit awkward.

I tried this config: log_level: debug log_debug_namespace_ignore: '^(?!z2m: MQTT)' log_debug_to_mqtt_frontend: true

The regexp seems to be correct (tested with https://regex101.com/ for ECMAscript regexps) but I'm still not seeing the messages. Something I'm doing wrong?

[LaurentChardin]

if i am not mistaken, it is mostly designed to work to filter the namespace. I think MQTT is not part of the namespace string but part of the message. Therefore it will not work. See code below:

    public debug(message: string, namespace: string = 'z2m'): void {
        if (this.level !== 'debug') {
            return;
        }
        if (this.debugNamespaceIgnoreRegex?.test(namespace)) {
            return;
        }

        this.logger.debug(message, {namespace});
    }

[johnholman]

Thanks, I see. Sounds like you can't use this to get just the MQTT publish messages then.

However ^(?!z2m) does seem to work to show only messages in the z2m namespace, which is probably sufficient.

If so suggest this example is added to the documentation as not obvious to people unfamiliar with regexps and maybe a common requirement. Also it might help to emphasize that the filtering is only on the namespace string.

[theDiverDK]

Please let us have logging back to how it was before the https://github.com/Koenkk/zigbee2mqtt/pull/22063 PR.

It's no problem, we have to enable it to come to the frontend.

Currently I am very glad I don't have a zigbee network issue, because I would not be able to debug it.

Please let us have the old 'info' back. Or create a new loglevel like it was before :-)

Thanks ;)

[Koenkk]

Will be addressed in https://github.com/Koenkk/zigbee2mqtt/pull/22619

[github-actions[bot]]

This issue is stale because it has been open 180 days with no activity. Remove stale label or comment or this will be closed in 30 days