Problem with v3.7.0 - all devices not available

[darth-aragoth]

I tried to switch from rospogrigio's localtuya v3.5. After rebooting all devices are not available. In log there are errors about requesting unknown DPS indices:

Logger: custom_components.localtuya.switch
Source: custom_components/localtuya/pytuya/__init__.py:304
Integration: LocalTuya integration (documentation, issues)
First occurred: 15:47:28 (162 occurrences)
Last logged: 15:48:25

[205...94b] Entity switch.gniazdko_rtv_1_parter is requesting unknown DPS index 1
[375...c64] Entity switch.gniazdko_rtv_sypialnia is requesting unknown DPS index 1
[834...291] Entity switch.gniazdko_rtv_salon_1 is requesting unknown DPS index 1
[834...2e3] Entity switch.gniazdko_raspberry_pi_4_ha is requesting unknown DPS index 1
[478...4cf] Entity switch.gniazdko_myszon_pc_2 is requesting unknown DPS index 1

Logger: custom_components.localtuya.sensor
Source: custom_components/localtuya/pytuya/__init__.py:304
Integration: LocalTuya integration (documentation, issues)
First occurred: 15:47:29 (221 occurrences)
Last logged: 15:50:59

[077...e6b] Entity sensor.czujnik_zewn_temp_i_wilg_bateria is requesting unknown DPS index 12
[205...467] Entity sensor.rtv_3_parter_prad is requesting unknown DPS index 4
[205...467] Entity sensor.rtv_3_parter_energia is requesting unknown DPS index 5
[375...edd] Entity sensor.komputer_3_prad is requesting unknown DPS index 4
[375...edd] Entity sensor.komputer_3_energia is requesting unknown DPS index 5

I successfully added my Zigbee gateway, but I can't add any sub-device. Errors from log:

Logger: custom_components.localtuya.config_flow
Source: custom_components/localtuya/config_flow.py:306
Integration: LocalTuya integration (documentation, issues)
First occurred: 15:57:07 (3 occurrences)
Last logged: 15:59:30

Unexpected exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/asyncio/locks.py", line 390, in acquire
    await fut
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/asyncio/tasks.py", line 456, in wait_for
    return fut.result()
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/config/custom_components/localtuya/config_flow.py", line 289, in validate_input
    detected_dps = await interface.detect_available_dps(data[CONF_DEVICE_ID])
  File "/config/custom_components/localtuya/pytuya/__init__.py", line 958, in detect_available_dps
    status = await self.status(cid)
  File "/config/custom_components/localtuya/pytuya/__init__.py", line 871, in status
    status = await self.exchange(DP_QUERY_NEW, cid=cid)
  File "/config/custom_components/localtuya/pytuya/__init__.py", line 824, in exchange
    msg = await self.dispatcher.wait_for(seqno, payload.cmd)
  File "/config/custom_components/localtuya/pytuya/__init__.py", line 504, in wait_for
    await asyncio.wait_for(self.listeners[seqno].acquire(), timeout=timeout)
  File "/usr/local/lib/python3.10/asyncio/tasks.py", line 458, in wait_for
    raise exceptions.TimeoutError() from exc
asyncio.exceptions.TimeoutError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/config/custom_components/localtuya/config_flow.py", line 457, in async_step_basic_sub_device_info
    self.dps_strings = await validate_input(self.hass, user_input)
  File "/config/custom_components/localtuya/config_flow.py", line 306, in validate_input
    await interface.close()
KeyError: (-100, <asyncio.locks.Semaphore object at 0x9a392850 [locked]>)

Logger: custom_components.localtuya.pytuya
Source: custom_components/localtuya/pytuya/__init__.py:504
Integration: LocalTuya integration (documentation, issues)
First occurred: 15:56:00 (4 occurrences)
Last logged: 15:59:30

[bfb...qpm] Failed to get status:
[bfb...qpm] Failed to get status for cid ##############:
[bfb...qpm] Failed to get status for cid ########:
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/asyncio/locks.py", line 390, in acquire
    await fut
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/asyncio/tasks.py", line 456, in wait_for
    return fut.result()
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/config/custom_components/localtuya/pytuya/__init__.py", line 976, in detect_available_dps
    status = await self.status()
  File "/config/custom_components/localtuya/pytuya/__init__.py", line 875, in status
    status = await self.exchange(DP_QUERY)
  File "/config/custom_components/localtuya/pytuya/__init__.py", line 824, in exchange
    msg = await self.dispatcher.wait_for(seqno, payload.cmd)
  File "/config/custom_components/localtuya/pytuya/__init__.py", line 504, in wait_for
    await asyncio.wait_for(self.listeners[seqno].acquire(), timeout=timeout)
  File "/usr/local/lib/python3.10/asyncio/tasks.py", line 458, in wait_for
    raise exceptions.TimeoutError() from exc
asyncio.exceptions.TimeoutError

Logger: custom_components.localtuya.pytuya
Source: custom_components/localtuya/pytuya/__init__.py:823
Integration: LocalTuya integration (documentation, issues)
First occurred: 15:56:10 (3 occurrences)
Last logged: 15:56:10

[bfb...qpm] Failed to get status: 'NoneType' object has no attribute 'write'
Traceback (most recent call last):
  File "/config/custom_components/localtuya/pytuya/__init__.py", line 976, in detect_available_dps
    status = await self.status()
  File "/config/custom_components/localtuya/pytuya/__init__.py", line 875, in status
    status = await self.exchange(DP_QUERY)
  File "/config/custom_components/localtuya/pytuya/__init__.py", line 823, in exchange
    self.transport.write(enc_payload)
AttributeError: 'NoneType' object has no attribute 'write'

I previously tested v3.6.5, but there no was v3.4 protocol (required for my Zigbee gateway).

[alexualbu]

@darth-aragoth, the PR above fixes for me 3.3 devices, 3.4 devices I still cannot add (similar error as yours). do you have only 3.4 devices?

[darth-aragoth]

No, most of my devices are version 3.3, a few are version 3.1. So far, only the Zigbee gateway has version 3.4.

[alexualbu]

@darth-aragoth, the changes which hopefully fix <3.4 devices are in 3.7.0-beta3 now if you want to give it a spin. @leeyuentuen, do you have devices on 3.4 to debug that or should i spend more time on it?

[leeyuentuen]

atm i have only a 3.4 gateway (was 3.3 but they update the firmware). but can't test it out atm.

[alexualbu]

ok, then I'll try to dig into it more this week. I only have some switches, but that should be enough for this.

[darth-aragoth]

I installed 3.7.0beta3. Most of my devices are working. I noticed, that I can't control my covers, when trying to change position:

Logger: homeassistant.components.websocket_api.http.connection
Source: custom_components/localtuya/cover.py:235
Integration: Home Assistant WebSocket API (documentation, issues)
First occurred: 13:03:00 (2 occurrences)
Last logged: 13:14:32

[2530108392] 'set_position_dp'
[2943011256] 'set_position_dp'
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 199, in handle_call_service
    await hass.services.async_call(
  File "/usr/src/homeassistant/homeassistant/core.py", line 1808, in async_call
    task.result()
  File "/usr/src/homeassistant/homeassistant/core.py", line 1845, in _execute_service
    await cast(Callable[[ServiceCall], Awaitable[None]], handler.job.target)(
  File "/usr/src/homeassistant/homeassistant/helpers/entity_component.py", line 213, in handle_service
    await service.entity_service_call(
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 686, in entity_service_call
    future.result()  # pop exception if have
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 961, in async_request_call
    await coro
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 726, in _handle_entity_call
    await result
  File "/config/custom_components/localtuya/cover.py", line 235, in async_close_cover
    COVER_MIN_POSITION, self._config[CONF_SET_POSITION_DP]
KeyError: 'set_position_dp'

And second error:

Logger: homeassistant.util.logging
Source: util/logging.py:168
First occurred: 13:03:00 (3 occurrences)
Last logged: 13:14:32

Exception in _update_handler when dispatching 'localtuya_45761564d8f15be413d3': ({'1': '2'},) Traceback (most recent call last): File "/config/custom_components/localtuya/common.py", line 710, in _update_handler self.status_updated() File "/config/custom_components/localtuya/cover.py", line 308, in status_updated if (self._state is not None) and (not self._device.is_connecting): AttributeError: 'TuyaDevice' object has no attribute 'is_connecting'
Exception in _update_handler when dispatching 'localtuya_45761564d8f15be413d3': ({'1': '3'},) Traceback (most recent call last): File "/config/custom_components/localtuya/common.py", line 710, in _update_handler self.status_updated() File "/config/custom_components/localtuya/cover.py", line 308, in status_updated if (self._state is not None) and (not self._device.is_connecting): AttributeError: 'TuyaDevice' object has no attribute 'is_connecting'

I successfully added my Zigbee gateway (v3.4), but still can't add any sub-device.

[alexualbu]

thanks, @darth-aragoth! I assume the covers are protocol <3.4. Unfortunately I don't have covers, so I'll take a look, but can't test so appreciate the logs. I think the 3.4 gateway can be added because the flow doesn't go into checking status or getting dps in general. We'll let you know when you can test a new version.

[alexualbu]

@darth-aragoth , I opened a PR which gets 3.4 devices working - I'd like to ask you to test it once @leeyuentuen pushes a new beta release as I don't have 3.4 gateways to check. Can you also confirm if your Covers are zigbee sub-devices or wifi ?

[darth-aragoth]

Hi. I tested beta5 release, I still cannot control my covers. My covers are wifi devices.

Logger: homeassistant.util.logging
Source: util/logging.py:168
First occurred: 15:49:41 (3 occurrences)
Last logged: 15:49:45

Exception in _update_handler when dispatching 'localtuya_45761564d8f15be413d3': ({'1': '2'},) Traceback (most recent call last): File "/config/custom_components/localtuya/common.py", line 724, in _update_handler self.status_updated() File "/config/custom_components/localtuya/cover.py", line 308, in status_updated if (self._state is not None) and (not self._device.is_connecting): AttributeError: 'TuyaDevice' object has no attribute 'is_connecting'
Exception in _update_handler when dispatching 'localtuya_45761564d8f15be413d3': ({'1': '3'},) Traceback (most recent call last): File "/config/custom_components/localtuya/common.py", line 724, in _update_handler self.status_updated() File "/config/custom_components/localtuya/cover.py", line 308, in status_updated if (self._state is not None) and (not self._device.is_connecting): AttributeError: 'TuyaDevice' object has no attribute 'is_connecting'

Logger: homeassistant.util.logging
Source: util/logging.py:168
First occurred: 15:49:41 (8 occurrences)
Last logged: 15:51:12

Exception in _update_handler when dispatching 'localtuya_45761564d8f15be413d3': ({'1': '2'},) Traceback (most recent call last): File "/config/custom_components/localtuya/common.py", line 724, in _update_handler self.status_updated() File "/config/custom_components/localtuya/cover.py", line 308, in status_updated if (self._state is not None) and (not self._device.is_connecting): AttributeError: 'TuyaDevice' object has no attribute 'is_connecting'
Exception in _update_handler when dispatching 'localtuya_45761564d8f15be413d3': ({'1': '3'},) Traceback (most recent call last): File "/config/custom_components/localtuya/common.py", line 724, in _update_handler self.status_updated() File "/config/custom_components/localtuya/cover.py", line 308, in status_updated if (self._state is not None) and (not self._device.is_connecting): AttributeError: 'TuyaDevice' object has no attribute 'is_connecting'
Exception in _update_handler when dispatching 'localtuya_45761564d8f15be413d3': ({'1': '1'},) Traceback (most recent call last): File "/config/custom_components/localtuya/common.py", line 724, in _update_handler self.status_updated() File "/config/custom_components/localtuya/cover.py", line 308, in status_updated if (self._state is not None) and (not self._device.is_connecting): AttributeError: 'TuyaDevice' object has no attribute 'is_connecting'

I also checked my Zigbee gateway, I still can't add any sub-device, I tried to set DPS manually, sub-device was added but unavailable. BTW which key value of subdevice should I put? For every sub-device I have "devId", "key" (both 22-character strings, probably always the same), "nodeId" and "uuid" (both 16-character strings, mostly, but not always the same). I always try both values, but none works.

Logger: custom_components.localtuya.config_flow
Source: custom_components/localtuya/config_flow.py:276
Integration: LocalTuya integration (documentation, issues)
First occurred: 16:00:36 (1 occurrences)
Last logged: 16:00:36

Unexpected exception
Traceback (most recent call last):
  File "/config/custom_components/localtuya/config_flow.py", line 457, in async_step_basic_sub_device_info
    self.dps_strings = await validate_input(self.hass, user_input)
  File "/config/custom_components/localtuya/config_flow.py", line 276, in validate_input
    if data.get(CONF_PARENT_GATEWAY):
AttributeError: 'str' object has no attribute 'get'

[Logger: homeassistant.util.logging
Source: util/logging.py:156
First occurred: 16:07:05 (1 occurrences)
Last logged: 16:07:05

Exception in _handle_sub_device_request when dispatching 'localtuya_gateway_bfb483f5d2c76aa333zqpm': ({'request': 'request_add', 'cid': 'e0798dfffec2f953', 'content': {'dps': {6: None}}},) Traceback (most recent call last): File "/usr/local/lib/python3.10/asyncio/locks.py", line 390, in acquire await fut asyncio.exceptions.CancelledError During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.10/asyncio/tasks.py", line 456, in wait_for return fut.result() asyncio.exceptions.CancelledError The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/config/custom_components/localtuya/common.py", line 410, in _handle_sub_device_request await self._get_sub_device_status(cid) File "/config/custom_components/localtuya/common.py", line 447, in _get_sub_device_status status = await self._interface.status(cid) File "/config/custom_components/localtuya/pytuya/__init__.py", line 884, in status status = await self.exchange(DP_QUERY_NEW, cid=cid) File "/config/custom_components/localtuya/pytuya/__init__.py", line 837, in exchange msg = await self.dispatcher.wait_for(seqno, payload.cmd) File "/config/custom_components/localtuya/pytuya/__init__.py", line 516, in wait_for await asyncio.wait_for(self.listeners[seqno].acquire(), timeout=timeout) File "/usr/local/lib/python3.10/asyncio/tasks.py", line 458, in wait_for raise exceptions.TimeoutError() from exc asyncio.exceptions.TimeoutError](`url`)

[alexualbu]

Thanks for checking @darth-aragoth! In that case for the covers I might have a quick fix. For the subdevices when adding them you need to use the cid / node_id that you get from the tuya devconsole

[alexualbu]

@darth-aragoth, can you test 3.7.0 beta8?

[darth-aragoth]

I successfully added Zigbee Gateway and first temperature / humidity sensor :) Great work, I will try to add other Zigbee and Bluetooth devices, thank you!

But there are still errors with covers:

Logger: homeassistant.components.websocket_api.http.connection
Source: custom_components/localtuya/cover.py:235
Integration: Home Assistant WebSocket API (documentation, issues)
First occurred: 19:35:56 (3 occurrences)
Last logged: 19:36:08

[2519786520] 'set_position_dp'
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 199, in handle_call_service
    await hass.services.async_call(
  File "/usr/src/homeassistant/homeassistant/core.py", line 1820, in async_call
    task.result()
  File "/usr/src/homeassistant/homeassistant/core.py", line 1857, in _execute_service
    await cast(Callable[[ServiceCall], Awaitable[None]], handler.job.target)(
  File "/usr/src/homeassistant/homeassistant/helpers/entity_component.py", line 216, in handle_service
    await service.entity_service_call(
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 798, in entity_service_call
    future.result()  # pop exception if have
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 977, in async_request_call
    await coro
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 838, in _handle_entity_call
    await result
  File "/config/custom_components/localtuya/cover.py", line 235, in async_close_cover
    COVER_MIN_POSITION, self._config[CONF_SET_POSITION_DP]
KeyError: 'set_position_dp'

Logger: homeassistant.util.logging
Source: util/logging.py:168
First occurred: 19:35:56 (6 occurrences)
Last logged: 19:36:08

Exception in _update_handler when dispatching 'localtuya_45761564d8f15be413d3': ({'1': '2'},) Traceback (most recent call last): File "/config/custom_components/localtuya/common.py", line 728, in _update_handler self.status_updated() File "/config/custom_components/localtuya/cover.py", line 308, in status_updated if (self._state is not None) and (not self._device.is_connecting): AttributeError: 'TuyaDevice' object has no attribute 'is_connecting'
Exception in _update_handler when dispatching 'localtuya_45761564d8f15be413d3': ({'1': '3'},) Traceback (most recent call last): File "/config/custom_components/localtuya/common.py", line 728, in _update_handler self.status_updated() File "/config/custom_components/localtuya/cover.py", line 308, in status_updated if (self._state is not None) and (not self._device.is_connecting): AttributeError: 'TuyaDevice' object has no attribute 'is_connecting'

[alexualbu]

Great! I don't believe the covers fix was pulled into 3.7.0 Are your covers 3.4 devices?

[darth-aragoth]

No, my covers are 3.3 devices.

[leeyuentuen]

@darth-aragoth I've just pulled a new beta version, could solve that is_connection problem

[darth-aragoth]

Errors from beta9:

Logger: homeassistant.components.websocket_api.http.connection
Source: custom_components/localtuya/cover.py:218
Integration: Home Assistant WebSocket API (documentation, issues)
First occurred: 09:14:29 (1 occurrences)
Last logged: 09:14:29

[2589476848] 'set_position_dp'
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 199, in handle_call_service
    await hass.services.async_call(
  File "/usr/src/homeassistant/homeassistant/core.py", line 1820, in async_call
    task.result()
  File "/usr/src/homeassistant/homeassistant/core.py", line 1857, in _execute_service
    await cast(Callable[[ServiceCall], Awaitable[None]], handler.job.target)(
  File "/usr/src/homeassistant/homeassistant/helpers/entity_component.py", line 216, in handle_service
    await service.entity_service_call(
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 798, in entity_service_call
    future.result()  # pop exception if have
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 977, in async_request_call
    await coro
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 838, in _handle_entity_call
    await result
  File "/usr/src/homeassistant/homeassistant/components/group/cover.py", line 201, in async_open_cover
    await self.hass.services.async_call(
  File "/usr/src/homeassistant/homeassistant/core.py", line 1820, in async_call
    task.result()
  File "/usr/src/homeassistant/homeassistant/core.py", line 1857, in _execute_service
    await cast(Callable[[ServiceCall], Awaitable[None]], handler.job.target)(
  File "/usr/src/homeassistant/homeassistant/helpers/entity_component.py", line 216, in handle_service
    await service.entity_service_call(
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 798, in entity_service_call
    future.result()  # pop exception if have
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 977, in async_request_call
    await coro
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 838, in _handle_entity_call
    await result
  File "/config/custom_components/localtuya/cover.py", line 218, in async_open_cover
    COVER_MAX_POSITION, self._config[CONF_SET_POSITION_DP]
KeyError: 'set_position_dp'

Logger: homeassistant
Source: custom_components/localtuya/cover.py:218
Integration: LocalTuya integration (documentation, issues)
First occurred: 09:14:30 (3 occurrences)
Last logged: 09:14:30

Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 977, in async_request_call
    await coro
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 838, in _handle_entity_call
    await result
  File "/config/custom_components/localtuya/cover.py", line 218, in async_open_cover
    COVER_MAX_POSITION, self._config[CONF_SET_POSITION_DP]
KeyError: 'set_position_dp'

[alexualbu]

@darth-aragoth , you seem to be missing the DP assignment for set_position from the config CONF_SET_POSITION_DP = "set_position_dp" Did you set this when creating the entity?

[darth-aragoth]

Now I understand the problem. My cover controllers have only one DP with value 1, 2 or 3 (move up, move down, stop, I don't know in what order). In config I have time-based positioning mode and "1_2_3" command set - that works fine with rospogrigio's localtuya version. At the beginning, I had to measure time needed to full-close each cover.

In config.yaml:

  - host: x.x.x.x
    device_id: xxx
    local_key: xxx
    friendly_name: Roleta 1
    protocol_version: "3.3"
    entities:
      - platform: cover
        friendly_name: Roleta 1
        id: 1
        commands_set: "1_2_3"
        positioning_mode: "timed"
        span_time: 20.8

In SmartLife config:

    "activeTime": 1623492697,
    "devId": "xxx",
    "displayOrder": 0,
    "dpMaxTime": 1679332060540,
    "dps": {
        "1": "3"
    },
    "errorCode": 0,
    "iconUrl": "https://images.tuyaeu.com/smart/icon/15524566468s3d5rsnldv_0.png",
    "isShare": false,
    "key": "xxx",
    "lat": "xxx",
    "localKey": "xxx",
    "lon": "xxx",
    "moduleMap": {
        "mcu": {
            "cadv": "",
            "isOnline": true,
            "verSw": "1.0.2"
        },
        "wifi": {
            "bv": "5.56",
            "cadv": "",
            "isOnline": true,
            "pv": "2.2",
            "verSw": "1.0.2"
        }
    },
    "name": "Rolety parter - sypialnia",
    "productId": "xxx",
    "resptime": 0,
    "runtimeEnv": "prod",
    "timezoneId": "Europe/Warsaw",
    "uuid": "xxx",
    "virtual": false

[darth-aragoth]

My cover controllers are similar to the one in the picture:

They only control the motor that raises and lowers the cover, but they have no feedback about the position of the cover, so it has to be time-controlled.

[alexualbu]

@darth-aragoth, i'll let @leeyuentuen comment as he seems to have made some changes to the cover vs rospogrigio's repo. I would suggest though that we move this to a different issues since we've digressed from the 3.7.0 topic.

[darth-aragoth]

I created new issue with time-based cover controllers support here: https://github.com/leeyuentuen/localtuya/issues/59