rospogrigio / localtuya

local handling for Tuya devices
GNU General Public License v3.0
2.93k stars 562 forks source link

Devices become unavailable #1117

Open fotis3d opened 1 year ago

fotis3d commented 1 year ago

Suddenly devices become unavailable forever. Tuya bulbs and plugs. In Tuya they work just fine. Reloading Local Tuya does not help.

To fix it I have to : 1) edit one of my devices (doesn't matter which one) without changing anything and then all the unavailable devices become available again. 2) Restart Home Assistant

BUT these work only for a short time. After a while the same thing happens.

Environment

CodyJon commented 1 year ago

Hmmm appears maybe so.. as last commit Nov.14.2022

Lets hope this can be ironed out, or someone else carriers the torch. Guess this motivates me more to replace the last few Tuya bulbs I have.

reidcooper commented 1 year ago

tbh Nov 14 2022 sounds like an active project to me... Unless we have the ability to make the change ourselves, we should be patient. Some of my projects take me months to get back to when I have the time or the willpower.

bobloadmire commented 1 year ago

tbh Nov 14 2022 sounds like an active project to me... Unless we have the ability to make the change ourselves, we should be patient. Some of my projects take me months to get back to when I have the time or the willpower.

I'm not saying anyone is entitled to anything. I'm just saying this issue is completely breaking the integration and I haven't seen anyone who actually contributes code to the project in the thread since Nov 7, so i'm just guessing it's abandoned. I fully concede that could not be the case

leeyuentuen commented 1 year ago

someone told me that the issue isn't visible on 3.5.x version but exists in 3.6.x. so if there is more logging data, maybe k can check the issue on it

https://github.com/leeyuentuen/localtuya

carlosCastro99 commented 1 year ago

I rolled back to 4.0.1 and no issues in the past 7 days

carlosCastro99 commented 1 year ago

But I think the internet connection drops has something to do with it, because 4.0.2 was also working fine till my internet connection start dropping.

agarbato commented 1 year ago

Same for me. Rolled back to 4.0.1 no issues in the past 6-7days. Wondering if 5.0.x just released could fix the issue.

bobloadmire commented 1 year ago

I'm going to wait for someone else to be the guinea pig

jeremyfry commented 1 year ago

I upgraded to 5.0.0 and I'm having the same issue.

bobloadmire commented 1 year ago

Very cool.

RayLation commented 1 year ago

I've tested a few versions, and never really found one that solved the probleme here. To me, it seems that localtuya has trouble recovering from network instabilities in general.

Granted I have some lights who are VERY sensitive to WI-FI interference. For example, when I start the micro-wave, ping latency becomes very high and I get lots of timeouts on my pings. But when the microwave stops, the ping latency goes back to normal and I would expect devices to reconnect and entities to come back to normal... but localtuya connection never actually recovers and entities stay "unavailable" until I manually force something. Integration reload sometimes works but hass core restart is often needed...until next network instability!

Loeffelmaster commented 1 year ago

I tried every version down to 4.0.1 now and none of the devices came back online after rebooting. :( When I enable internet and DNS for those devices again, they are immediately back online, but as far as I read you should disable access to internet and DNS for these devices.

Anycubic commented 1 year ago

I tried every version down to 4.0.1 now and none of the devices came back online after rebooting. :( When I enable internet and DNS for those devices again, they are immediately back online, but as far as I read you should disable access to internet and DNS for these devices.

As far as I understood if you block internet access permanently, after a while those devices will go in zombie state, like you are saying

rlause commented 1 year ago

I have just come across this thread. I started using local tuya only weeks ago (with ECO-DIM.07 wifi). From the very beginning devices dropped randomly after some hours/days having had internet access disabled for them. Enabling internet access brings them back. Disabling internet access again will then lead to further drops. So local tuya for me seems to require internet access of the tuya devices at the moment. It is clearly no local solution.

Is there a plan to fulfill the claim of a really local tuya integration?

reidcooper commented 1 year ago

I have just come across this thread. I started using local tuya only weeks ago (with ECO-DIM.07 wifi). From the very beginning devices dropped randomly after some hours/days having had internet access disabled for them. Enabling internet access brings them back. Disabling internet access again will then lead to further drops. So local tuya for me seems to require internet access of the tuya devices at the moment. It is clearly no local solution.

Is there a plan to fulfill the claim of a really local tuya integration?

@rlause I would recommend downgrading your installation to version 4.0.2. Next, I believe the only internet required is upon initial startup if you configure your Tuya devices to fetch the keys from the cloud. This is explained in the README. I'd suggest downgrading and fetching the Tuya keys manually and adding them to your device configurations in Local Tuya. That should move it to be really local, from my understanding.

There seems to be a general consensus with the current releases where devices drop after a period of time. Myself included.

rootd commented 1 year ago

I upgraded to 5.0.0 and I'm having the same issue.

I can confirm that this is still an issue in 5.0.0

Loeffelmaster commented 1 year ago

I tried every version down to 4.0.1 now and none of the devices came back online after rebooting. :( When I enable internet and DNS for those devices again, they are immediately back online, but as far as I read you should disable access to internet and DNS for these devices.

As far as I understood if you block internet access permanently, after a while those devices will go in zombie state, like you are saying

After running 4.0.1 for some time, all devices came back online and all devices stay online.

Only blocking the internet access will make those devices going zombie, you also must make sure, that they can't access your DNS (most times the router), which is in my case AdGuard Home on my HA. I put in a blacklist for the Tuya devices. Everything works fine and also comes back after restarting HA or powering it down. That means they aren't allowed to get a message back from the DNS. Not resolving the address will still make them go zombie.

I still do not understand how Local Tuya decides if a device is offline. As I found out something very strange.

If you have one of these "offline" devices (ping works) with an online device in one group and you turn on or off this group ALL devices will turn on or off. Even those that are offline. So they can't be offline as they are clearly accepting commands. But even though they are reacting and fulfilling these commands, they won't go online afterwards.

manunited10 commented 1 year ago

I have the same issue. Devices go randomly unavailable and stay unavailable mostly. Even updating to the latest (v5.0.0) had no effect. I will downgrade to lower versions as suggested above to see what happens. I should mention that, when device is unavailable, I can still control the device using tinytuya python package.

pvillanyi commented 1 year ago

I went back to 4.0.1 it seams to be better. Let's observe it.

Olivier-Bette commented 1 year ago

Hello I'm on v5.0.0. I had no issue since the release of that version. Until a few days ago. 9 of my 43 tuya bulbs and switches became unavailable 1-2 hours after a restart of Home Assistant. I tried this workaround. Go to parameters - devices and services - entities section. Filter on "localtuya" then select and delete all unavailable devices marked with a red "!". Reboot Home Assistant. I did this today and since then all my devices remain available. If this can help some of you. Regards

CodyJon commented 1 year ago

Still have large amount of issues on new rollout. Will revert back to 4.0.1 as suggested.

pvillanyi commented 1 year ago

With the new update 2023.2.1 the unknown state flipping is back, also with 4.0.1

rlause commented 1 year ago

@Loeffelmaster wrote:

Only blocking the internet access will make those devices going zombie, you also must make sure, that they can't access your DNS (most times the router), which is in my case AdGuard Home on my HA. I put in a blacklist for the Tuya devices. Everything works fine and also comes back after restarting HA or powering it down. That means they aren't allowed to get a message back from the DNS. Not resolving the address will still make them go zombie.

I tried the trick with blocking DNS additionally to blocking internet access for the Tuya devices. Along the way I stay with the most current Local Tuya version. For quite a long time and regardless of several HA restarts the Tuya devices are still displayed "online".

I am not sure that this will continue to work for some more weeks (or forever). However, the behaviour already seems to be better than before. I'll report again.

filipeaparicio91 commented 1 year ago

To anyone still having this issue, this is likely happening if you or someone else in your house has the Tuya app installed. Tuya devices only support 1 local connection at a time, so if your device reports to the Tuya app it will become unavailable in HA.

To test this:

  1. Uninstall the Tuya app from your phone (alternatively you can force close + delete cache)
  2. Unplug / cut power from the device for 10 seconds
  3. Power the device again
  4. Reload the Local Tuya integration and your device should become available again.

To prevent this from happening again, uninstall the Tuya app since every time you restart HA, your Tuya devices might change the local connection back to your phone again.

If this works for you, please leave a comment so others can see.

bobloadmire commented 1 year ago

If that's the case, why does rolling back versions fix the issue?

filipeaparicio91 commented 1 year ago

@bobloadmire, not always. Many reported that although it temporarily solved the issue, devices started becoming unavailable again after a few days. Because of the single local connection limitation, a device could easily have connected again with HA after a roll-back, and then switched back to the app connection. Basically you have the devices jumping back and forth between HA and the Tuya app. Even if you keep HA running uninterrupted, a DCHP DNS lease usually lasts 24/48h depending on your network configuration, meaning that unless you attributed static IPs to your devices (IP reservations are not the same), the IP change can be enough for the device to momentarily lose connection to HA and connect to the Tuya app instead. I was having the same issue in the past and did not have to roll-back to solve it.

Either way, this obviously doesn't apply to every case - if a user does not have the Tuya app installed, the issue clearly lies somewhere else. But for those who the device unavailability happens at random, the most logical reason would be this.

bobloadmire commented 1 year ago

@bobloadmire, not always. Many reported that although it temporarily solved the issue, devices started becoming unavailable again after a few days. Because of the single local connection limitation, a device could easily have connected again with HA after a roll-back, and then switched back to the app connection. Basically you have the devices jumping back and forth between HA and the Tuya app. Even if you keep HA running uninterrupted, a DCHP DNS lease usually lasts 24/48h depending on your network configuration, meaning that unless you attributed static IPs to your devices (IP reservations are not the same), the IP change can be enough for the device to momentarily lose connection to HA and connect to the Tuya app instead. I was having the same issue in the past and did not have to roll-back to solve it.

Either way, this obviously doesn't apply to every case - if a user does not have the Tuya app installed, the issue clearly lies somewhere else. But for those who the device unavailability happens at random, the most logical reason would be this.

Well it's interesting because just today I installed a new tuya wall switch, added it via the app, then added it to HA via this integration and everything works fine from both HA and the tuya app simultaneously. I was unaware I shouldn't be using both at the same time. But it also doesn't seem to be cause issues, presumably because I'm rolled back to 4.0.2

filipeaparicio91 commented 1 year ago

@bobloadmire, when you are able to use HA and Tuya simultaneously, it's because the Tuya app is communicating with the device via Cloud and not locally - that's why the Tuya app always maintains connection. Local Tuya on the other hand doesn't have a cloud connection, so once the local access is lost, the device becomes unavailable. This is not documented in this specific Repo, but there are others that wrote this on their integrations (Tuya Local for example mentions this problem).

filipeaparicio91 commented 1 year ago

@bobloadmire this link has very helpful information regarding some common errors that occur with Tuya devices: https://github.com/make-all/tuya-local

Also, regarding the single local connection limitation, there are some devices from Tuya that support more that one, I'm just not sure which, and there isn't a lot of information about it online. This could also be the reason why only some devices become unavailable.

manunited10 commented 1 year ago

@filipeaparicio91 while your comment makes sense, does that mean, when a device gets disconnected from home-assistant's localtuya, and connects to the tuya app, then it will reject localtuya's attempts? If that's true, how's that I was able to easily connect to (and control) the unavailable device through tiny-tuya's python package? For me as I mentioned a couple of comments above, even upgrading to v5 didn't work and eventually downgrade to v4.0.1 fixed the issue so far.

filipeaparicio91 commented 1 year ago

@manunited10 What I'm saying is that based on my tests, some Tuya devices struggle jumping back and forth between TCP connections - this is even mentioned in tiny-tuya's documentation as well:

Tuya devices only allow one TCP connection at a time. Make sure you close the TuyaSmart or SmartLife app before using TinyTuya to connect.

I was able to consistently replicate the unavailable status issue by trying to control a lightbulb from both HA and the mobile app, to a point where it would stop receiving commands from HA but carry-on working on the Tuya app.

The light would some times stay unavailable just for a few minutes, and in other instances it became unavailable indefinitely. Curiously enough, when unavailable, the bulb was unable to receive commands from HA, but was able to report it's attribute changes (ex. brightness) - although they would not appear in the logbook. As soon as I clear the Tuya mobile app cache and force close it, reloading the Local Tuya integration re-establishes the connection consistently.

Now, if you ask me why downgrading works, or why you were able to control your devices through Tiny Tuya - I have no idea. It apparently works for some users (even if temporarily) and it doesn't for others. Tuya devices behave in unexpected ways, and the only consistent behavior that I was able to achieve was by getting rid of the Tuya mobile app.

Loeffelmaster commented 1 year ago

Also, regarding the single local connection limitation, there are some devices from Tuya that support more that one, I'm just not sure which, and there isn't a lot of information about it online. This could also be the reason why only some devices become unavailable.

If you block the access to the internet for your Tuya devices and block the access the the DNS too (the device is not allowed to get any answer from the DNS), then that device is invisible for Tuya. In Tuya it goes offline and will never come back there. I have the Tuya integration and local Tuya running. This is my setup process to get devices into Local Tuya.

  1. Normally install the device in Tuya and get the data neccessary for integration into Local Tuya
  2. Change the IP from the device to an IP in the blocked IP range (device will get offline in Tuya), you need to force the device to reload the DHCP settings. Either you turn off power for that device and turn it back on, or you network management system can force it to do so be disconnecting it WLAN connection (possible e.g. in Unifi)
  3. Change the name of this device in HA in the Tuya integration (I add Tuya in front, for the why see below)
  4. Integrate the device into local with all settings and name it.

Now the device is availabe in Local Tuya, but not in Tuya. Tuya doesn't even know the IP address of the device and so it can't address it. It wouldn't be able anyway, as those connection run over the internet.

Still I have a device that becomes offline for Local Tuya from time to time. It still can be turned on and of and set via a group even though Local Tuya states the device as offline. So this is no doubt an error in the program of Local Tuya, as the device is there and can be controlled even though Local Tuya states otherwise.

How the regain control of an offline device in Local Tuya? There is a way to make these devices online in Local Tuya again without giving them access to the internet nor any DNS. That this is possible just shows, that there must be something wrong in Local Tuya. Perhaps Local Tuya misses a device response, sets it to offline and then never tries it again until it get a new response of the device? I don't know. It is strange as power off and power on the device normally doesn't change the state. So how to get it online again, if that clearly doesn't work? For setup purposes most Tuya devices can also be accessed via Bluetooth. So if you activate Bluetooth on your smartphone or tablet and then start the Tuya app, you can try to control the device. The device will be shown as offline. If you try to send several commands to the device it will come online after a few tries. In that second the device reacts to the command from the Tuya app it gets online again in Local Tuya too!! Now you can close the Tuya app again and/or disable Bluetooth. The device is working as before in Local Tuya.

I have Bluetooth devices via Tuya and so I need the Tuya integration for them. Nothing changes, even when looking in Tuya itself and querying the data for the device it will show (if there is still any shown) the old IP which was used to setup the device in Tuya.

filipeaparicio91 commented 1 year ago

@Loeffelmaster

If you block the access to the internet for your Tuya devices and block the access the the DNS too (the device is not allowed to get any answer from the DNS), then that device is invisible for Tuya.

This basically achieves the same as removing the Tuya mobile app, so it makes sense if this setup works for you.

Still I have a device that becomes offline for Local Tuya from time to time. It still can be turned on and of and set via a group even though Local Tuya states the device as offline. So this is no doubt an error in the program of Local Tuya, as the device is there and can be controlled even though Local Tuya states otherwise.

Do you experience this regardless of the Local Tuya version? If this happens to a single device, the problem could lie there - some Tuya devices are clearly more problematic than others.

So if you activate Bluetooth on your smartphone or tablet and then start the Tuya app, you can try to control the device. The device will be shown as offline. If you try to send several commands to the device it will come online after a few tries. In that second the device reacts to the command from the Tuya app it gets online again in Local Tuya too!!

This is probably because the Bluetooth connection does not use TCP, meaning it that in theory, the connection between the device and Local Tuya is not broken - I'm just guessing here.

Loeffelmaster commented 1 year ago

Do you experience this regardless of the Local Tuya version? If this happens to a single device, the problem could lie there - some Tuya devices are clearly more problematic than others.

Yes it is only one device but I can't figure out why. I have more bulbs like that in use and none of them shows this behavior. Even other bulbs from the same type connected to the same AP doesn't have this problem. I'm thinking of changing it, perhaps something is wrong with it.

This is probably because the Bluetooth connection does not use TCP, meaning it that in theory, the connection between the device and Local Tuya is not broken - I'm just guessing here.

It isn't broken for sure. It is a bug in Local Tuya. As described above the bulb is shown offline and you can't turn it on or off directly, but when you trigger a group where this bulb is in and you turn the bulb on or off with other bulbs together it is triggered. To be even stranger, when you have a group where only this bulb is in, then it is as if you trigger the bulb directly.

When it come to the Local Tuya version, I'm down to 4.0.1 in that version I have only this single bulb that goes offline from time to time and most times comes back online after a few hours. In versions above 4.0.1 more devices show this kind of problem.

I haven't tested the newest 5.1.0-beta0 by now. Hadn't had the time as I had major problems with my PC after installing it new because of a program corrupting my network connection by opening thousands of ports. Needed 2 weeks to find it and I have installed my computer several time the last 3 weeks because of that.

Acrobot commented 1 year ago

For anyone having this issue: I have seemingly narrowed it down to the cloud integration. I did not have problems, then when I added cloud integration I started experiencing the devices becoming unavailable. Finally, after many tries, I am on 5.1.0-beta0 and the only thing that helped was removing the cloud account. The devices have been stable so far.

cloudbr34k84 commented 1 year ago

For anyone having this issue: I have seemingly narrowed it down to the cloud integration. I did not have problems, then when I added cloud integration I started experiencing the devices becoming unavailable. Finally, after many tries, I am on 5.1.0-beta0 and the only thing that helped was removing the cloud account. The devices have been stable so far.

What do you mean remove the cloud account? Do you mean official tuya integration from HA?

Acrobot commented 1 year ago

@cloudbr34k84 So you can set up the cloud API configuration in this integration so it automatically reads your device name and rotates the local key here:

https://github.com/rospogrigio/localtuya-homeassistant/raw/master/img/9-cloud_setup.png

However, if you click "Do not configure cloud API account", you won't have those functions, but at least in my experience it fixes the problem with devices becoming unavailable.

GTunney commented 1 year ago

I don’t think this is an issue with the app and local tuya clashing in terms of connections numbers. My entire family have deleted the SmartLife app from our phones so only HA is accessing tuya and we’re still getting devices going unavailable. Only seemed to start happening in the last month or 2 I’d say. Reloading integration doesn’t seem to work. It requires a restart of HA.

filipeaparicio91 commented 1 year ago

@GTunney are you only using the Local Tuya integration, or do you also have the official Tuya integration installed? Also, are you able to check if the local key for the unavailable devices is still the same?

GTunney commented 1 year ago

@GTunney are you only using the Local Tuya integration, or do you also have the official Tuya integration installed? Also, are you able to check if the local key for the unavailable devices is still the same?

Yes local tuya only.

I’ll need to jump on the laptop and check the local key. If it was local key would it still work after a restart with the old key?

filipeaparicio91 commented 1 year ago

If it was local key would it still work after a restart with the old key?

No it wouldn't. Do your logs show any errors? Also, are the devices that become unavailable always the same?

GTunney commented 1 year ago

No errors that I can see within the logs.

It does seem to be the same small selection of devices playing up.

filipeaparicio91 commented 1 year ago

@GTunney, did you see any improvement after uninstalling the Tuya mobile apps? Have these devices always been behaving this way?

GTunney commented 1 year ago

Afraid not. Seems to be just as many switches to unavailable. 1 or 2 have had general issues in the past dropping off but the worst offender a smart plug has only recently started happening and it’s a relatively new device. Seems to be no pattern at all either. 9383C666-EDD6-4523-9463-370754B02268

filipeaparicio91 commented 1 year ago

@GTunney it looks like that device has connectivity issues. Is it located near a router? If not, can you try moving it closer to your router for a few hours to check if the behaviour changes?

fotis3d commented 1 year ago

No solution on this yet ? Anyone ? Strange we are still on 4.0.1 in order to work correctly.

pvillanyi commented 1 year ago

I've started to test tuya local from make-all. I had to create the device profile for my plugs, but it works good.

sympapa commented 1 year ago

I have the same problem as @GTunney only on certain lighting devices. Sometimes it becomes unavailable and takes a few seconds to a few minutes to come back. I also started testing tuya-local@make-all custom integration and do not have the same problem there.

cataseven commented 1 year ago

just delete official smart life app from all of your devices. this app is working on the background and query your devices.

cloudbr34k84 commented 1 year ago

Don't you need the devices added Plus some devices don't have issue