search

fairecasoimeme / ZiGate

Zigate is an Universal Zigbee Gateway
http://zigate.fr
171 stars 59 forks source link

Somes Xiaomi devices lose after shunt down #91

Closed ricky074 closed 5 years ago

ricky074 commented 5 years ago

I'm on Domoticz with Zigate plugin. When I have to power off my USB Zigate (power cut for example) some Xiaomi devices does'nt work any more after restart. I have to re appair them each time. (door sensor and temp/hum/press sensor) It's a litle bit annoying... Does anybody has the same issue ? ( I am on 3.0e firmware) Thanks

pipiche38 commented 5 years ago

This looks very weird as I was under the assumption that a short power-off/power-on of the Zigate wouldn't impact the commission part !

z3tof commented 5 years ago

Same problem..

Yesterday, I restarted my rapsberry pi with zigate and just after that, I lost 5 xiaomi devices (only 2 devices weren't lost).

I had to re appair them...

I am on 3.0e firmware too.

pipiche38 commented 5 years ago

@z3tof are you also on Domoticz with Zigate plugin ?

Could you also tell us more on what your network is made off ? Only battery based objects or do you have main powered one ?

jordisalichs commented 5 years ago

Hi, First of all, I'm not usign ZiGate hardware. But I use the same meshreen module. All end-devices are Xiaomi battery powered at less than 5 meters. I was usign original firmware from NXP (ZigbeeNodeControlBridge_JN5168.bin) for about 4 o 5 months without major problems. This weekend I flashed to ZiGate and I have the same problems explained here losing end-devices in few hours. ZiGate use the same NXP ZigBee Stack that original firmware. Why this lossing?

jordisalichs commented 5 years ago

Hi again, NXP firmware that not have the losing problem is JN-AN-1223-ZigBee-IoT-Gateway-Control-Bridge-v1014, that use SDK JN-SW-4168. By other hand, ZiGate use JN-SW-4170. I think that the problem could be this new stack version. I tried to compile ZiGate with JN-SW-4168, but I cannot get it. Does anybody know if it possible or can try?

pipiche38 commented 5 years ago

I don't have much historic, but I never hear this problem on loosing devices when shutting down for a short time the Zigate. They were some issue of loosing Xiaomi devices much it is was on runtime .

If the issue introduced by 3.0e ?

jordisalichs commented 5 years ago

I detect the problem only on runtime. (The message posted in a wrong issue) I only tested the 3.0e.

doudz commented 5 years ago

For me, I got the problem at runtime with firmware 3.0d and after shutdown with 3.0e

jordisalichs commented 5 years ago

Hi, I flashed ZigBee module with JN-AN-1223-ZigBee-IoT-Gateway-Control-Bridge-v1014 firmware and until now any device is lost. I think that the problem is with JN-SW-4170 Stack.

KiwiHC16 commented 5 years ago

Test I did today and observations:

From a PDM erased Zigate, then start network, then inclusion of a few Xiaomi devices (x11), plus one Ikea Remote, plus 3 bulbs (Ikea, GLEDOPTO). All worked fine for the night.

Zigate USB unplugged left unplugged for 2 minutes, plug it back and restarted Jeedom plugin (re-establish USB connection).

Lost all Xiaomi. Kept all the rest.

Xiaomi behavior: • data request which Ack if zigate present • if Zigate not answering device continue Data Request • The Zigate is back receive data request and do a Leave with rejoin set to 1 but device do not rejoin.

Remote Ikea: • data request which Ack if zigate present • if Zigate not answering device start sending beacon request • When the Zigate is back, reply to beacon and then device do rejoin

Bulb (routeurs) Ikea and Gledopto: • devices don't do data request so no issue.

For me it's not a new behavior, I remember seeing it since the beginning, probably a year ago. It was seen as lots devices in run time.

My theory is: Xiaomi devices don't behavior as other devices. They send Data Request all the time what ever is happening. This is the root of the issue.

Case 1) Runtime: From zigate point of view, when it receive the DataRequest it answer with an Ack. But if the zigate do not receive the Data Request it starts removing the device after a while. When suddently it receive a Data Request again, it reply with a Leave with Rejoin. But Xiaomi devices don't rejoin. Device lost. Case 2) Restart Zigate: devices are not in Zigate anymore, so at first Data Request, send back a Leave with Rejoin. But Xiaomi devices don't rejoin. Device lost.

As the timer was increased a lot by Akila, case 1 should not be seen anymore (or less often). But case 2 is not mitigated by the timer.

As we can't change the firmware of Xiaomi NE, we need to find a nice idea to change the firmware.

I will try an other test to confirm my Theory which is to attach a Xiaomi Device to a routeur and switch off the Zigate. I'm ready to bet that I will not loose the Xiaomi Device.

I'll keep you posted.

KiwiHC16 commented 5 years ago

Just finished the test. Re-associated a Xiaomi Device to a router (GledOpto bulb). Data tranfer to Zigate, no issue. Unplugged Zigate from USB, wait 2 minutes, Clic on Device button, data sent to routeur, router can't fnid zigate. Normal. Plug again, re-start plugin (usb coonection) Clic on Device button, data sent to routeur, router sent to zigate. Normal.

For each clic there is a data request before but no leave.

Xiomi Device is not lost, CQFD.

Now the question is how to solve the issue.

pipiche38 commented 5 years ago

@KiwiHC16 , @jordisalichs I think that if we want to have a conclusive test, @jordisalichs could you kindly use the same protocol as @KiwiHC16 with the JN-AN-1223-ZigBee-IoT-Gateway-Control-Bridge-v1014 firmware, otherwise I'm afraid that we won't be able to take any conclusion at all.

KiwiHC16 commented 5 years ago

An other test, join a xaiomi device on zigate, unplug, do lots of device button press which trigger data reqest and ZCL reports, plug, restart plugin, press button and no Leave sent by Zigate... no device lost.. ( few routers and one xiaomi device on the network).

All tests since the beginning are with 3.0.e firmware.

jordisalichs commented 5 years ago

I will try to compile JN-AN-1223-ZigBee-IoT-Gateway-Control-Bridge-v1014 (from NXP) using JN-SW-4170 Stack instanteof JN-SW-4168 Stack. Is it possible to compile ZiGate with JN-SW-4168 Stack? Yesterday I can not do it.

fairecasoimeme commented 5 years ago

Hi , I'll study all of this but for information : JN-SW-4168 Stack correspond to Zigbee Home Automation 1.2 JN-SW-4170 Stack is the Zigbee 3.0

jordisalichs commented 5 years ago

Yesterday I tried to compile JN-AN-1223-ZigBee-IoT-Gateway-Control-Bridge-v1014 using JN-SW-4170 and I cannot get it. I think that it is possible, but I do not how do it. Usign JN-AN-1223-ZigBee-IoT-Gateway-Control-Bridge-v1014 no Xiaomi end device are lost in 2 days.

I think that ZiGate is based on JN-AN-1216-Zigbee-3-0-IoT-ControlBridge-v1011, is it correct? I wil try to flash my module with image included in zip from NXP. If after flash Xiaomi devices are not lost, the problem is at ZiGate modifications (some parameter changed in zcl_options.h) I will try to do this night.

max5962 commented 5 years ago

I really don't understand how xiaomi works... I unplug my zigate to test somethings => lost 6 devices ( maybe more ) i need to test.

It's maybe stupid, bug there is a way to log data received from unbind device ? It could help the user to automaticaly detect that usecase and re-bind it .

jordisalichs commented 5 years ago

After some heavy testing usign JN-AN-1223-ZigBee-IoT-Gateway-Control-Bridge-v1014 firmware any device is lost. But I can only add 10 nodes. I cannot add any more. I do not know if a RAM, EEPROM o firmware limitation.

fairecasoimeme commented 5 years ago

Perhaps the problem is due to free space in Flash or EEprom. I have to search into the config file : ZigbeeNodeControlBridge.zpscfg In fact, since the new sdk, the new bin size is bigger and may be I must decrease the devices number capacity. I'll search ...

Thanks for returns

jordisalichs commented 5 years ago

I think that application cannot write in FLASH, only in EEPROM. Exist one parameter for PDM (EXTERNAL_FLASH) that use external flash ic to save all PDMs. I soldered a compatible FLASH memory, but I get an error when compile. I asked to NXP community but not response yet.

max5962 commented 5 years ago

Does issues-91 and issues-89 have the same root-cause ? There is a zigate competitor : https://projetsdiy.fr/zigbee2mqtt-projet-de-pont-domotique-zigbee-vers-mqtt-compatible-xiaomi-aqara/ Maybe they do not have the same problem ( maybe the solution is in the project ? )

fairecasoimeme commented 5 years ago

Hi, I think i found the problem. It's seem to be a memory flash problem... You can try the following binary in aim to confirm fix : https://github.com/fairecasoimeme/ZiGate/blob/v3.0f/Module%20Radio/Firmware/src/ZiGate/Build/bin/ZiGate_3.0f_pre_release.bin

Please tell me if it's OK.

Edit : You must erase EEprom...

pipiche38 commented 5 years ago

@fairecasoimeme which problem is related to the memory flash issue, is that Device lost when rebooting/powering-off the Zigate, or is that the general one of loosing Xiaomi devices ?

is the 3.0f_pre_release includes also the uint24bits ?

Hydci commented 5 years ago

La version 3.0F pre release a bien résolu le problème pour ma part plus de capteur qui réponde plus même après un reboot

deennoo commented 5 years ago

3.0f pre-release a resolut aussi les soucis de leave de mon coté

ricky074 commented 5 years ago

Ok c'est pas trop galère pour passer en 3.0f ? Il faut ré-appairer tous les devices ? Au niveau du plugin, faut-il repartir de 0 ?

ricky074 commented 5 years ago

@fairecasoimeme : sais-tu dans combien de temps devrait sortir la release 3.0f ?

fairecasoimeme commented 5 years ago

Salut, la release devrait sortir fin du mois. Il faudra faire un Erase EEprom et ré-appairer tous les devices. J'espère que c'est la dernière fois que la structure de données changent. J'espère que je trouverai comment mieux maitriser les données ...

A bientôt

pipiche38 commented 5 years ago

Ok c'est pas trop galère pour passer en 3.0f ? Il faut ré-appairer tous les devices ? Au niveau du plugin, faut-il repartir de 0 ?

Salut Ricky074, si tu utilises le Plugin ZIgate pour Domoticz, tu n'auras qu'a refaire l'appairage avec la Zigate fraîchement effacée, par contre coté plugin et Domoticz tu ne touches à rien, tout va se reconnecter.

ISO-B commented 5 years ago

Can someone confirm that 3.0f fixed this problem?

fairecasoimeme commented 5 years ago

yep 3.0f fix the problem. After shutdown, there are no device loosed

tekaeren commented 5 years ago

It is strange, as in https://github.com/fairecasoimeme/ZiGate/issues/38 I am still loosing xiaomi devices during restart.