ZBInfo does not list all devices

Agnes56fr commented 5 months ago

PROBLEM DESCRIPTION

A clear and concise description of what the problem is.

REQUESTED INFORMATION

Make sure your have performed every step and checked the applicable boxes before submitting your issue. Thank you!

[x] Read the Contributing Guide and Policy and the Code of Conduct
[ ] Searched the problem in issues
[ ] Searched the problem in discussions
[x] Searched the problem in the docs
[ ] Searched the problem in the chat
[ ] Device used (e.g., Sonoff Basic): _____
[x] Tasmota binary firmware version number used: _____
- [x] Pre-compiled
- [ ] Self-compiled
[ ] Flashing tools used: _____
[ ] Provide the output of command: Backlog Template; Module; GPIO 255:
```
Configuration output here:
```

- [ ] If using rules, provide the output of this command: `Backlog Rule1; Rule2; Rule3`:
```lua
  Rules output here:

[ ] Provide the output of this command: Status 0:
```
STATUS 0 output here:
```

- [ ] Set `weblog` to 4 and then, when you experience your issue, provide the output of the Console log:
```lua
  Console output here:

TO REPRODUCE

Steps to reproduce the behavior:

EXPECTED BEHAVIOUR

A clear and concise description of what you expected to happen.

SCREENSHOTS

If applicable, add screenshots to help explain your problem.

ADDITIONAL CONTEXT

Add any other context about the problem here.

(Please, remember to close the issue when the problem has been addressed)

I'm facing a strange problem with a sonoff Zibgbee bridge flashed with Tasmota 13.4.0.4 12 devices appear in the main menu if I make a command zbinfo 1, Zbinfo 2, ... up to 12 everything works If I make zbinfo then only 11 devices are listed

any advice ?

Agnes56fr commented 5 months ago

This is the result of Zbinfo command

10:39:48.387 RSL: SENSOR = {"ZbInfo":{"0x4067":{"Device":"0x4067","Name":"leak_spa","IEEEAddr":"0xA4C138A01BB7384C","ModelId":"TS0207","Manufacturer":"_TZ3000_ocjlo4ea","Endpoints":[1],"Config":["A01.4"],"ZoneType":42,"ZoneStatus":0,"Reachable":false,"BatteryPercentage":27,"LastSeen":6245,"LastSeenEpoch":1712735743,"LinkQuality":0}}} 10:39:48.398 RSL: SENSOR = {"ZbInfo":{"0x4DEA":{"Device":"0x4DEA","Name":"temp_etage","IEEEAddr":"0xA4C13853E0677A42","ModelId":"TS0201","Manufacturer":"_TZ3000_fllyghyj","Endpoints":[1],"Config":["T01"],"Temperature":21.63,"Humidity":47.1,"Reachable":true,"BatteryPercentage":100,"BatteryLastSeenEpoch":1712740344,"LastSeen":1644,"LastSeenEpoch":1712740344,"LinkQuality":58}}} 10:39:48.409 RSL: SENSOR = {"ZbInfo":{"0xD1A7":{"Device":"0xD1A7","Name":"Temp_Exterieure","IEEEAddr":"0xA4C13870440D3D7E","ModelId":"TS0201","Manufacturer":"_TZ3000_fllyghyj","Endpoints":[1],"Config":["T01"],"Temperature":13.92,"Humidity":80.4,"Reachable":true,"BatteryPercentage":57,"BatteryLastSeenEpoch":1712740254,"LastSeen":534,"LastSeenEpoch":1712741454,"LinkQuality":52}}} 10:39:48.419 RSL: SENSOR = {"ZbInfo":{"0x43B0":{"Device":"0x43B0","Name":"temp_ssol","IEEEAddr":"0xA4C138EB643973FB","ModelId":"TS0201","Manufacturer":"_TZ3000_fllyghyj","Endpoints":[1],"Config":["T01"],"Temperature":15.01,"Humidity":71.64,"Reachable":true,"BatteryPercentage":17,"BatteryLastSeenEpoch":1712740734,"LastSeen":18,"LastSeenEpoch":1712741970,"LinkQuality":97}}} 10:39:48.430 RSL: SENSOR = {"ZbInfo":{"0xD76C":{"Device":"0xD76C","Name":"Temp_SalleSport","IEEEAddr":"0xA4C138CE7343D647","ModelId":"TS0201","Manufacturer":"_TZ3000_xr3htd96","Endpoints":[1],"Config":["T01"],"Temperature":19.94,"Humidity":43.14,"Reachable":true,"BatteryPercentage":100,"BatteryLastSeenEpoch":1712740649,"LastSeen":1338,"LastSeenEpoch":1712740650,"LinkQuality":50}}} 10:39:48.443 RSL: SENSOR = {"ZbInfo":{"0x7493":{"Device":"0x7493","Name":"Leak_arriere_cuisine","IEEEAddr":"0xA4C138A2E5BE0A8F","ModelId":"TS0207","Manufacturer":"_TZ3000_kstbkt6a","Endpoints":[1],"Config":["A01.4"],"ZoneType":42,"ZoneStatus":0,"Reachable":false,"BatteryPercentage":9,"LastSeen":10672,"LastSeenEpoch":1712731316,"LinkQuality":71}}} 10:39:48.455 RSL: SENSOR = {"ZbInfo":{"0xF49F":{"Device":"0xF49F","Name":"Temp_Salon","IEEEAddr":"0xA4C138263243EF05","ModelId":"TS0201","Manufacturer":"_TZ3210_huzkzqyk","Endpoints":[1,2],"Config":["~01.1","I01","T02"],"Illuminance":0,"Temperature":20.3,"Humidity":50.1,"Reachable":true,"BatteryPercentage":100,"BatteryLastSeenEpoch":1712739920,"LastSeen":17,"LastSeenEpoch":1712741971,"LinkQuality":55}}} 10:39:48.466 RSL: SENSOR = {"ZbInfo":{"0x56D6":{"Device":"0x56D6","Name":"_Prise_1","IEEEAddr":"0x144405602BD5B370","ModelId":"TS011F","Manufacturer":"_TZ3000_okaz9tjs","Endpoints":[1],"Config":["O01","P01"],"Power":0,"RMSVoltage":229,"ActivePower":0,"Reachable":true,"LastSeen":12,"LastSeenEpoch":1712741976,"LinkQuality":50}}} 10:39:48.477 RSL: SENSOR = {"ZbInfo":{"0x2EAF":{"Device":"0x2EAF","Name":"_Prise_2","IEEEAddr":"0x164705602BD5B370","ModelId":"TS011F","Manufacturer":"_TZ3000_okaz9tjs","Endpoints":[1],"Config":["O01","P01"],"Power":0,"RMSVoltage":230,"ActivePower":0,"Reachable":true,"LastSeen":12,"LastSeenEpoch":1712741976,"LinkQuality":71}}} 10:39:48.488 RSL: SENSOR = {"ZbInfo":{"0x5A70":{"Device":"0x5A70","Name":"_Prise_3","IEEEAddr":"0xD04805602BD5B370","ModelId":"TS011F","Manufacturer":"_TZ3000_okaz9tjs","Endpoints":[1],"Config":["O01","P01"],"Power":0,"RMSVoltage":237,"ActivePower":0,"Reachable":true,"LastSeen":12,"LastSeenEpoch":1712741976,"LinkQuality":97}}} 10:39:48.499 RSL: SENSOR = {"ZbInfo":{"0x9332":{"Device":"0x9332","Name":"_Prise_4","IEEEAddr":"0xAE4B05602BD5B370","ModelId":"TS011F","Manufacturer":"_TZ3000_okaz9tjs","Endpoints":[1],"Config":["P01","O01"],"RMSVoltage":230,"ActivePower":0,"Power":0,"Reachable":true,"LastSeen":13,"LastSeenEpoch":1712741975,"LinkQuality":31}}} 10:39:48.506 RSL: RESULT = {"ZbInfo":"Done"}

the first device is not listed but appear with "zbinfo 1" 10:41:08.085 CMD: zbinfo 1 10:41:08.094 RSL: SENSOR = {"ZbInfo":{"0xE404":{"Device":"0xE404","Name":"leak_pompe_piscine","IEEEAddr":"0xA4C13889F8BBF6A5","ModelId":"TS0207","Manufacturer":"_TZ3000_t6jriawg","Endpoints":[1],"Config":["A01.4"],"ZoneType":42,"ZoneStatus":0,"Reachable":false,"BatteryPercentage":9,"LastSeen":8929,"LastSeenEpoch":1712733139,"LinkQuality":53}}} 10:41:08.101 RSL: RESULT = {"ZbInfo":"Done"}

arendst commented 5 months ago

How/Where do you execute the command?

If on the web console it is most likely the cyclic log buffer is full while adding the twelvth entry deleting the first. Try the same using a syslog server.

Agnes56fr commented 5 months ago

First, Thank you for answering

the command is executed via http http://192.168.0.129/cm?cmnd=zbinfo

{"ZbInfo":{"0x4067":{"Device":"0x4067","Name":"leak_spa","IEEEAddr":"0xA4C138A01BB7384C","ModelId":"TS0207","Manufacturer":"_TZ3000_ocjlo4ea","Endpoints":[1],"Config":["A01.4"],"ZoneType":42,"ZoneStatus":0,"Reachable":true,"BatteryPercentage":75,"BatteryLastSeenEpoch":1712742948,"LastSeen":175,"LastSeenEpoch":1712742948,"LinkQuality":0}},"ZbInfo":{"0x4DEA":{"Device":"0x4DEA","Name":"temp_etage","IEEEAddr":"0xA4C13853E0677A42","ModelId":"TS0201","Manufacturer":"_TZ3000_fllyghyj","Endpoints":[1],"Config":["T01"],"Temperature":21.97,"Humidity":47.21,"Reachable":true,"BatteryPercentage":100,"BatteryLastSeenEpoch":1712740344,"LastSeen":979,"LastSeenEpoch":1712742144,"LinkQuality":58}},"ZbInfo":{"0xD1A7":{"Device":"0xD1A7","Name":"Temp_Exterieure","IEEEAddr":"0xA4C13870440D3D7E","ModelId":"TS0201","Manufacturer":"_TZ3000_fllyghyj","Endpoints":[1],"Config":["T01"],"Temperature":14.11,"Humidity":80.78,"Reachable":true,"BatteryPercentage":57,"BatteryLastSeenEpoch":1712740254,"LastSeen":1070,"LastSeenEpoch":1712742053,"LinkQuality":55}},"ZbInfo":{"0x43B0":{"Device":"0x43B0","Name":"temp_ssol","IEEEAddr":"0xA4C138EB643973FB","ModelId":"TS0201","Manufacturer":"_TZ3000_fllyghyj","Endpoints":[1],"Config":["T01"],"Temperature":15.01,"Humidity":71.64,"Reachable":true,"BatteryPercentage":17,"BatteryLastSeenEpoch":1712740734,"LastSeen":1153,"LastSeenEpoch":1712741970,"LinkQuality":97}},"ZbInfo":{"0xD76C":{"Device":"0xD76C","Name":"Temp_SalleSport","IEEEAddr":"0xA4C138CE7343D647","ModelId":"TS0201","Manufacturer":"_TZ3000_xr3htd96","Endpoints":[1],"Config":["T01"],"Temperature":20.03,"Humidity":43.25,"Reachable":true,"BatteryPercentage":100,"BatteryLastSeenEpoch":1712740649,"LastSeen":671,"LastSeenEpoch":1712742452,"LinkQuality":52}},"ZbInfo":{"0x7493":{"Device":"0x7493","Name":"Leak_arriere_cuisine","IEEEAddr":"0xA4C138A2E5BE0A8F","ModelId":"TS0207","Manufacturer":"_TZ3000_kstbkt6a","Endpoints":[1],"Config":["A01.4"],"ZoneType":42,"ZoneStatus":0,"Reachable":false,"BatteryPercentage":9,"LastSeen":11807,"LastSeenEpoch":1712731316,"LinkQuality":71}},"ZbInfo":{"0xF49F":{"Device":"0xF49F","Name":"Temp_Salon","IEEEAddr":"0xA4C138263243EF05","ModelId":"TS0201","Manufacturer":"_TZ3210_huzkzqyk","Endpoints":[1,2],"Config":["~01.1","I01","T02"],"Illuminance":0,"Temperature":20.4,"Humidity":52,"Reachable":true,"BatteryPercentage":100,"BatteryLastSeenEpoch":1712739920,"LastSeen":89,"LastSeenEpoch":1712743034,"LinkQuality":55}},"ZbInfo":{"0x56D6":{"Device":"0x56D6","Name":"_Prise_1","IEEEAddr":"0x144405602BD5B370","ModelId":"TS011F","Manufacturer":"_TZ3000_okaz9tjs","Endpoints":[1],"Config":["O01","P01"],"Power":0,"RMSVoltage":231,"ActivePower":0,"Reachable":true,"LastSeen":6,"LastSeenEpoch":1712743117,"LinkQuality":58}},"ZbInfo":{"0x2EAF":{"Device":"0x2EAF","Name":"_Prise_2","IEEEAddr":"0x164705602BD5B370","ModelId":"TS011F","Manufacturer":"_TZ3000_okaz9tjs","Endpoints":[1],"Config":["O01","P01"],"Power":0,"RMSVoltage":232,"ActivePower":0,"Reachable":true,"LastSeen":5,"LastSeenEpoch":1712743118,"LinkQuality":65}},"ZbInfo":{"0x5A70":{"Device":"0x5A70","Name":"_Prise_3","IEEEAddr":"0xD04805602BD5B370","ModelId":"TS011F","Manufacturer":"_TZ3000_okaz9tjs","Endpoints":[1],"Config":["O01","P01"],"Power":0,"RMSVoltage":239,"ActivePower":0,"Reachable":true,"LastSeen":6,"LastSeenEpoch":1712743117,"LinkQuality":97}},"ZbInfo":{"0x9332":{"Device":"0x9332","Name":"_Prise_4","IEEEAddr":"0xAE4B05602BD5B370","ModelId":"TS011F","Manufacturer":"_TZ3000_okaz9tjs","Endpoints":[1],"Config":["P01","O01"],"RMSVoltage":230,"ActivePower":0,"Power":0,"Reachable":true,"LastSeen":2,"LastSeenEpoch":1712743121,"LinkQuality":34}},"ZbInfo":"Done"}

I do not know what a syslog server is can you explain ?

Agnes56fr commented 5 months ago

for more informations zbstatus list all 12 devices

{"ZbStatus1":[{"Device":"0xE404","Name":"leak_pompe_piscine"},{"Device":"0x4067","Name":"leak_spa"},{"Device":"0x4DEA","Name":"temp_etage"},{"Device":"0xD1A7","Name":"Temp_Exterieure"},{"Device":"0x43B0","Name":"temp_ssol"},{"Device":"0xD76C","Name":"Temp_SalleSport"},{"Device":"0x7493","Name":"Leak_arriere_cuisine"},{"Device":"0xF49F","Name":"Temp_Salon"},{"Device":"0x56D6","Name":"_Prise_1"},{"Device":"0x2EAF","Name":"_Prise_2"},{"Device":"0x5A70","Name":"_Prise_3"},{"Device":"0x9332","Name":"_Prise_4"}]}

arendst commented 5 months ago

Executing using HTTP or on the web GUI will result in the same experience; the report buffer is full and loses the first entry in favour of the last.

A syslog server is a device able to log messages from all kinds of resources like your Zigbee bridge.

You'll have to learn with the fact that executing the command will not report all devices. You already found the solution by reporting individual devices.

barbudor commented 5 months ago

Syslog is a standard protocol to send log messages over the network to a receiving log server The web console as a limited buffer so @arendst is assuming that it is too short to get all results from the ZbInfo command Syslog would not have that limitation as each log is immediately sent to the server

If you are running linux, syslog is most of the time already running and listenning on port 514 (see https://tasmota.github.io/docs/Troubleshooting/#syslog-logging) You could configure tasmota to send the logs to that server using: backlog loghost xxx.xxx.xxx.xx; logport 514; syslog 2

If you are running windows and have Python install, you can run a small syslog equivalent with this python code which use port 5140, so configure tasmota with: backlog loghost xxx.xxx.xxx.xx; logport 5140; syslog 2

Agnes56fr commented 5 months ago

ok, can it be a buffer size problem if the result come from a http request ?

arendst commented 5 months ago

can it be a buffer size problem if the result come from a http request

The buffer size is indeed the cause BUT ... The buffer size is bound to available RAM. Increasing the buffer wil decrease precious RAM. That's a trade-off I have to make. Increasing the buffer is not the solution as you can imagine; having fifty zigbee devices would need all available RAM. You have to change the way you retreive your data like you already found out by individual ZbInfo commands.

sfromis commented 5 months ago

Commands via http is the worst way if you want to see full results. You'll generally get better results in the Tasmota web UI console, as the JavaScript code in the browser can save data it got before it rolled out of the limited buffer space in Tasmota itself. More generally, make sure to already be in the web UI console when the command is issued.

However, for a command producing a lot of output at the same time, the lines may roll off the buffer before being picked up by the browser; the default refresh interval is 2.345 seconds.

If you use serial console instead, you'll generally get all lines as they are produced, without the limitations incurred when accessing results via the web UI.

I'd say that the best strategy is what Tasmota is designed for, using MQTT. The results will be published, without losing lines, and can be picked up that way.

Jason2866 commented 5 months ago

Closing since answered and it is not an issue.

Agnes56fr commented 5 months ago

"You'll generally get better results in the Tasmota web UI console, as the JavaScript code in the browser can save data it got before it rolled out of the limited buffer space in Tasmota itself."

When in the web UI it does exactly the same

Using MQTT would probably better, however, it require to have something else when asking the Zigbeebrige with zbinfo would (should?) provide the same result.

I do not really understand this buffer problem. you mean that when we execute a command in the console or through a http request, if the answer is to long they you might receive only a part of it ?

Then it should be truncated in the middle of something, wich is not what happen here. brackets are still properly open and closed

s-hadinger commented 5 months ago

I will eventually try eventsource which should eliminate this problem with the console

Agnes56fr commented 5 months ago

hello, thank you for your answer, what is eventsource ?

Agnes56fr commented 5 months ago

?m=1 output is a lot longer than /cm/cmnd=zbinfo and ?m=1 is not truncated

arendst commented 5 months ago

Then it should be truncated in the middle of something, wich is not what happen here. brackets are still properly open and closed

Indeed! There has been a lot of work in providing the data as safe as possible. In fact what happens is that every device is a seperate line in the log buffer. If a new line comes in with a length longer than available space in the buffer the first line is removed. Hence you'll see a nicely constructed JSON message.

sfromis commented 5 months ago

When in the web UI it does exactly the same

I already told you why. But if the log spans over seconds, you can get a lot more by already being in the console.

The story is not about what you can receive, but about the method you use to retrieve it. The web UI console polls the Tasmota buffer at intervals, by default 2.345 seconds, meaning that commands (like ZbInfo) producing a lot of output will only have the last lines (not bytes) fitting into the buffer when polled.

Use serial, syslog or MQTT. Just not the web UI console.

EventSource is a protocol allowing devices like Tasmota to push data to a browser session, instead of the browser polling. Currently not implemented in Tasmota, except for the GPIO Viewer.

Agnes56fr commented 5 months ago

Thank you all for trying to help me.

I use http://192.168.0.129/cm?cmnd=zbinfo in a php file and analyse the answer to display the status of all devices in a html page (which is static, it does not update as device's status change, it just update when you refresh the page, wich is more than enough considering it display temperatures mostly).

I understood why it could not cut the result anywhere since the size of the buffer is checked device/device I though I understood that being in the console might create the buffer limit problem, then though it was finally preferable to be in the console (so I'm a bit lost on understanding the buffer problem really)

using eventsource to push devices' changes to a web page sound great, but since it is not implemented in Tasmota, I will not go this way.

instead, I'm about to rewrite my php in order to pull info devices/devices with zbinfo 1, 2, 3... I thought it was better to make one global request but if you say I can only if it is short enough then I will go device/device.

Agnes56fr commented 5 months ago

with zbinfo 1, 2, 3, 4... it does work

Thanks a lot for all your answers

sfromis commented 5 months ago

Tasmota has an internal buffer limited to a "moderate" number of bytes/lines, which is what you see when entering the console page. If you then leave it open, the JavaScript code in the browser console will poll the Tasmota buffer at intervals, default 2.345 seconds, and add any new lines to the browser buffer, which can grow very long over time.

The one-line response from http command ZbInfo is what it added to the buffer, but only collected when the command execution ended, and therefore cannot have more than the available buffer size. Hence my statement about using http commands being the worst way.

Therefore it is "preferable" to be in the console, unless you issue a command (like ZbInfo) able to overfill the Tasmota buffer much quicker than the polling interval. Still, the straightforward approach is to use serial, syslog or MQTT. Tasmota is not designed for http to provide a solid API, after all, the "M" in the name means "MQTT".

Agnes56fr commented 5 months ago

Thanks again for all those informations.

So the ZBInfo command overfilled the internal tasmota buffer. I also understood that MQTT should be the way.

For me having a MQTT server beside tasmota is kinda double sources of informations since tasmota already keep the informations and since I can easyly access it with the http request even if not designed for that and all that without having to contact the devices, well, for my application, I could not see any good reason to have another server with MQTT on it.

I thank you very much all for your work and for your answers

arendst / Tasmota