search

dresden-elektronik / deconz-rest-plugin

deCONZ REST-API plugin to control ZigBee devices
BSD 3-Clause "New" or "Revised" License
1.89k stars 498 forks source link

NEO Tuya Smart Zigbee Temperature And Humidity Sensor Detect Luminous Intensity #4790

Closed nunokaneco closed 2 years ago

nunokaneco commented 3 years ago

Device

https://s.click.aliexpress.com/e/_AKXsy9

Screenshots

image

The device is detected in the deconz -> Phoscon -> add devices -> sensores

But I don't have any entity created in the Home Assistant and the device over the Phoscon devices doesn't show any info about the sensor also only the name is available as you can see the picture bellow
image

Please let me know if you need any further information about this device

Thank you for your support and best regards

Smanar commented 3 years ago

Heu yep, ca we have the same screenshoot but with better resolution ? ^^ It seem there is a new cluster, but I m not sure for the value (0xE002 ?)

Your device have a standard cluster for illumination, but I think all others values are from a manufacture cluster, and I don't know this one ...

nunokaneco commented 3 years ago

hoot but with better resolution ? ^^

image

Please let me know if it is good for you now

Thank you and best regards

Smanar commented 3 years ago

Ok, so congratulation, you have find a new chinese cluster ... Nothing about this one on other project.

BTW you don't have zigbee sniffer ?

Are you able to compile something to test (need a linux machine with full OS) ? Perhaps this cluster use the same protocol than the EF00.

nunokaneco commented 3 years ago

on't have

Hello Sorry I don't have any zigbee sniffer just the usb pen conbee...

And yes I have the possibility to test/ run something in linux machine but you need to let me know the steps that i have to execute in this case

Thank you

Smanar commented 3 years ago

You have the procedure here You have the procedure here https://github.com/dresden-elektronik/deconz-rest-plugin#install-deconz-development-package-optional-linux-only

The modification is easy

        case TUYA_CLUSTER_ID:
            // Tuya manfacture cluster:
            handleTuyaClusterIndication(ind, zclFrame);
            break;

Just add

        case TUYA_CLUSTER_ID:
        case 0xE002:
            // Tuya manfacture cluster:
            handleTuyaClusterIndication(ind, zclFrame);
            break;

It will be "hacky", but if it work you will see in log, lines starting by

Tuya debug 4.................

Else, we will need someone sniff this cluster to decode it.

Pedder007 commented 3 years ago

HeyHo, same here 😄 Got two of these device (Ali, Poland) today and ioBroker only is able to fetch the following: grafik

Besides this, I also cannot change anything at that device in Phoscon. Neither the name nor I can delete it. The second one has been added this way: grafik

As I would have here my spare PI3B with a dedicated ConBeeII stick on it and deCONZ already installed, I could give it a try, but never did that bevor. Thus I need a bit more help how to.

  1. Installation of the development package seems to be easy (haven't done it yet)
  2. in which file (and directory) would I have to make the modification?
  3. Where can I find the log-file? - I flipped through all directories below '/usr/share/deCONZ/....' but couldn't find it

Please, a bit more concrete 😄 If the rain might stay as announced I will hopfully find the time .... 🔨

Smanar commented 3 years ago

With the same new cluster ?

1 - Yes, it the first line, "sudo apt install deconz-dev", remember to install deconz first https://phoscon.de/en/conbee2/install#raspbian 2 - where you want, can use defaut path, the last command will replace the good file 3 - If you have the GUI, logs can be found in deconz/help, else you need to run deconz using command line https://github.com/dresden-elektronik/deconz-rest-plugin/wiki/deCONZ-debug-flags

And lot of rain here too ^^

Pedder007 commented 3 years ago

Yes also with 'E002 unknown'. Sorry for coming back late, but had to tackle something further stuff, indoor .... Thanks for your hints, But I'm still confused 😄

Regarding question (2) I meant where I would have to put case TUYA_CLUSTER_ID: case 0xE002: in?

Regarding (3), I do not have a directory named 'help' in my deconz directory. And in the others there is nothing called [somehwat].log

image

Smanar commented 3 years ago

The modification is in file de_web_plugin.cpp file around line 990

To access log it s in the application deconz, on the menu "help"

Pedder007 commented 3 years ago

@Smanar, the first problem occured: 'sudo apt install deconz-dev' gives back: 'E: Paket deconz-dev kann nicht gefunden werden.' Means deconz-dev package not found. Any ideas on this? deCONZ itself, including the Rest-plugin package are installed definitly as I already used the system for other test purposes. And I also stopped the deCONZ service.

Edit: Aha! Just found #2811, which helped to get that issue tackled 👍

Smanar commented 3 years ago

including the Rest-plugin package are installed definitly

So you have used the command described here "Import Phoscon public key" and "Configure the APT repository for deCONZ" https://phoscon.de/en/conbee2/install ?

If yes it mean your environnement haven't the package, what is your OS ?

Pedder007 commented 3 years ago

yes, yes, as deconz was already running on that Raspi incl. Phoscon and the connect to Pimatic :-)

The installation of deconz-dev now is ready See my comment above regarding #2811

Pedder007 commented 3 years ago

Hm, I cannot find the file 'de_web_plugin.cpp' I searched in the whole directory '/usr/share/deCONZ' incl. sub-directories.

Pedder007 commented 3 years ago

I'm running buster on a Raspi 3B And deCONZ incl. the Rest-API are definetly running :-) grafik

and deconz-dev has also been installed propperly: version (2.11.04-raspbian-buster-beta)

Pedder007 commented 3 years ago

well, found the file 'de_web_plugin.cpp' now on the Git deconz-rest-plugin web-page (repository), but I have no clue how to continue. I will re-install/-compile now the rest-plugin ....

Pedder007 commented 3 years ago

@Smanar, ok got it done now. I have no clue if the file has been there before, but I now found it in 'home/pi/deconz-rest-plugin' I also found the respective line 991. So next question, befor the next step as I wasn't able to delete the device before already from my prod environment. Shall I pair the device via Phoscon before the change in the file or afterwards?

Pedder007 commented 3 years ago

@Smanar, simply gave it a try now that way:

  1. added the line 'case 0xE002:' in the file
  2. started deCONZ (gui)
  3. started Phoscon + paired the device
  4. went back to deCONZ gui and opened the log/debug view

By choosing 'HTTP' I found the following regarding the device: "2021-05-17T21:07Z","manufacturername":"_TZ3000_qaaysllp","modelid":"TS0201","name":"LightLevel 3","state":{"dark":false,"daylight":false,"lastupdated":"2021-05-17T21:06:05.147","lightlevel":18196,"lux":66},"type":"ZHALightLevel","uniqueid":"84:71:27:ff:fe:fe:7e:a7-01-0400"}}

By choosing 'INFO_L2' and covering the light sensor, this was shown: 22:22:06:012 APS-DATA.indication from child 0x4C91 22:22:06:014 Node data 0x847127fffefe7ea7 profileId: 0x0104, clusterId: 0x0400 22:22:06:016 0x847127FFFEFE7EA7: update ZCL value 0x01/0x0400/0x0000 after 0 s 22:22:06:018 enqueue event state/lightlevel for /sensors/3 22:22:06:019 enqueue event state/lastupdated for /sensors/3 22:22:06:021 enqueue event state/dark for /sensors/3 22:22:06:022 enqueue event state/lux for /sensors/3 22:22:06:029 payload: 000021c30b 22:22:06:030 binding for attribute reporting of ep: 0x01 cluster 0x0001 seems to be active 22:22:06:031 binding for attribute reporting of ep: 0x01 cluster 0x0400 seems to be active 22:22:06:079 Websocket 192.168.188.79:54266 send message: {"e":"changed","id":"3","r":"sensors","state":{"dark":true,"daylight":false,"lastupdated":"2021-05-17T21:22:06.018","lightlevel":3011,"lux":2},"t":"event","uniqueid":"84:71:27:ff:fe:fe:7e:a7-01-0400"} (ret = 2125086960) 22:22:06:127 discard sensor state push for 3: state/lastupdated (already pushed) 22:22:06:147 discard sensor state push for 3: state/dark (already pushed) 22:22:06:169 discard sensor state push for 3: state/lux (already pushed)

I also checked all other 'debug Levels' but wasn't able to find anything in additon regarding the device (847127FFFEFE7EA7) or maybe the new cluster (E002).

And I also wasn't able to find your above mentioned 'Tuya debug 4.................'

I close for today. What might be next steps?

Pedder007 commented 3 years ago

Really last info for today. I checked the incoming sensor data by 'http://192.168.xxx.xx:xxxx/api/xxxxxxxx/sensors:

3  
config  
battery 100
on true
reachable true
tholddark 12000
tholdoffset 7000
ep 1
etag "7143226ac82f8dbac515d5decff3a846"
lastseen "2021-05-17T22:44Z"
manufacturername "_TZ3000_qaaysllp"
modelid "TS0201"
name "LightLevel 3"
state  
dark false
daylight false
lastupdated "2021-05-17T21:22:14.082"
lightlevel 17710
lux 59
type "ZHALightLevel"
uniqueid "84:71:27:ff:fe:fe:7e:a7-01-0400"
Smanar commented 3 years ago

Hello, so much tests ^^.

For the moment the other sensor will be not created, because I don't know how the device send data, so you will be not able to see something in the api. The modification is just to enable tuya logging, but if you don't see in log line starting by "Tuya debug" (with just "info" and "info_l2") it mean this device don't use standard tuya request.

Sorry, we need someone decode the requests used by this device :(

Pedder007 commented 3 years ago

Hi, I tried my best 😃 Sorry to hear that this needs more technical expertise. It's really a cool device, as I was searching for something in combination (lux, temp and humidity) having also a display. And it seems sending data in relatively short cycles. I currently do have another thermometer running in the living room (also with display). But it partly only sends its data after 15-20 minutes and this is too long for a meaningful shutter-steering for sun-protection ... Looking forward then, that someone from the developers might get such a device for implementation.

Pedder007 commented 3 years ago

Maybe one addition: To delete the device from one's environment, use the 'delete node' function via the deCONZ gui. Via Phoscon it doesn't work.

Smanar commented 3 years ago

There is someone with the same device than you, sensor with display, but more standard https://github.com/dresden-elektronik/deconz-rest-plugin/issues/4886

Looking forward then, that someone from the developers might get such a device for implementation.

Or someone in an other zigbee project, I have take a look on google, ZHA have a similar one, but without the cluster 0xE002

Pedder007 commented 3 years ago

I was not awaiting, that one might come to the idea to hide temp and humidity somewhere in a dark corner in a device 🤣, that no one can find it any longer 🤣 , but surely they all would like to sell their own gateways and send all data then to China 😆 PS. the link the the other thread seems not working any more

Smanar commented 3 years ago

Lol. But zigbee move a lot, changes are fast.

And I have just tried the link again, it work on my side ....

Pedder007 commented 3 years ago

Yes that's true, but I had the hope that basic sensor data might not be changed that much, ... but however regarding gateways and gathering data ... ;-)

This link? https://s.click.aliexpress.com/e/_9HESSD I tested again and kicked out now 's.', then it re-directs and works.

Saw the device now remembered to it, but choosed the other (here discussed) as I found the design better...., sh... happens ... ;-) Ahh, and the others have been cheaper in addition ...

github-actions[bot] commented 3 years ago

As there has not been any response in 21 days, this issue has been automatically marked as stale. At OP: Please either close this issue or keep it active It will be closed in 7 days if no further activity occurs.

nunokaneco commented 3 years ago

Lol. But zigbee move a lot, changes are fast.

And I have just tried the link again, it work on my side ....

Hello I just try again without success can you please let me know what is the version in yo HA home assistant plugin/fw/etc version are you running ?

Thank you

Pedder007 commented 3 years ago

Hi, you mean my environment? I tested it on/with: Raspi 4B, Buster ConBee II, DeCONZ (ioBroker and Pimatic) However, the temperature and humidity data currently is not ‚accessible‘ by deCONZ. The basic data and the illumination (lux) is accessible. Do you see a chance to get this new cluster to become readable?

Pedder007 commented 3 years ago

HiHo developers 😄 Nobody out there taking care for this device? 😢 can I do something in addtion?

Smanar commented 3 years ago

Someone have sniffed and decripted the new cluster https://github.com/zigpy/zha-device-handlers/issues/862 I will try to take a look on it tommorow, but seriously if there is too much problem I will give up, we can't use all "magic" cluster possible, and more if they are used only for one device.

Pedder007 commented 3 years ago

Thanks a lot @Smanar 👍 I surely hope, that no really challenging problems will come up 😄 - press thumbs!!!

Vebryn commented 3 years ago

0xe002 handle specific device functions like alarms (cf. https://github.com/zigpy/zha-device-handlers/issues/862#issuecomment-847357919). Temperature and humidity are reported into endpoint 2 in standard cluster. Endpoint 2 is not announced through discovering process.

Pedder007 commented 3 years ago

read through the thread you linked in, but understood maybe only one third of it. Did I got the essence right, that temp and humidity is send by this device in a way that it cannot be processed by ConBee/deCONZ?

Smanar commented 3 years ago

The endpoint is not visible, so deconz don't use it.

Can try this code, it will force device creation, but not sure the api will use them

sudo apt install deconz-dev
git clone --branch tuya_test5 https://github.com/Smanar/deconz-rest-plugin.git
cd deconz-rest-plugin
qmake && make -j2
sudo cp ../libde_rest_plugin.so /usr/share/deCONZ/plugins
Pedder007 commented 3 years ago

Hey @Smanar, I just tested what you wrote, incl. renaming my original 'deconz-rest-plugin' directory and cloning yours to my Raspi. But it didn't change anything. I checked it in the deCONZ gui, where I was not able to read anything from the device. I only got red blinking at the device symbole, when I tried reading something (incl. the basic cluster). Afterwards I connected my deCONZ/ConBee II test environment to ioBroker and checked it there. Here I saw the sensor, but still only the objects I had before: image Beyond this, the developers in zigpy/zha-device-handlers#862 seems making some further progress. Maybe in the end they will find the key ...!?

Smanar commented 3 years ago

The change will not be visible in the GUI, the device is not zigbee compliant. But If I m right you will have some other sensor on the endpoint 0x02, in the API. You need to compile the code, replace the lib file, and re-incude the device (no need to delete the previous one)

You probably have the red blinking because the device is sleeping.

Pedder007 commented 3 years ago

ok, I just restarted my test deconz Raspi and checked the sensors by:

http://192.xxx.xxx.xxx:8080/api/xxxxxxxx/sensors

but didn't get back anything. (not reachable, whereas the same works for my prod deconz)

During the 'qmake && make -j2', in the end two errors occured, maybe this is also interesting for you:

g++ -c -pipe -Wno-attributes -Wno-psabi -Wall -O2 -std=gnu++1y -Wall -W -D_REENTRANT -fPIC -DDECONZ_DLLSPEC=Q_DECL_IMPORT -DSKIP_EMPTY_PARTS=QString::SkipEmptyParts -DUSE_WEBSOCKETS -DHAS_SQLITE3 -DHAS_OPENSSL -DGW_SW_VERSION=\"2.12.00\" -DGW_SW_DATE=1624099948 -DGW_API_VERSION=\"1.16.0\" -DGIT_COMMMIT=\"353b346572fad83a309bbd8f2b461b38fedb51a5\" -DGW_AUTO_UPDATE_AVR_FW_VERSION=0x260b0500 -DGW_AUTO_UPDATE_R21_FW_VERSION=0x26420700 -DGW_MIN_AVR_FW_VERSION=0x26390500 -DGW_MIN_R21_FW_VERSION=0x26660700 -DGW_MIN_DERFUSB23E0X_FW_VERSION=0x22030300 -DGW_DEFAULT_NAME=\"Phoscon-GW\" -DQT_NO_DEBUG -DQT_PLUGIN -DQT_WIDGETS_LIB -DQT_GUI_LIB -DQT_SERIALPORT_LIB -DQT_WEBSOCKETS_LIB -DQT_NETWORK_LIB -DQT_CORE_LIB -I. -I../.. -I../../common -isystem /usr/include/arm-linux-gnueabihf/qt5 -isystem /usr/include/arm-linux-gnueabihf/qt5/QtWidgets -isystem /usr/include/arm-linux-gnueabihf/qt5/QtGui -isystem /usr/include/arm-linux-gnueabihf/qt5/QtSerialPort -isystem /usr/include/arm-linux-gnueabihf/qt5/QtWebSockets -isystem /usr/include/arm-linux-gnueabihf/qt5/QtNetwork -isystem /usr/include/arm-linux-gnueabihf/qt5/QtCore -Irelease -I. -I/usr/lib/arm-linux-gnueabihf/qt5/mkspecs/linux-g++ -o release/firmware_update.o firmware_update.cpp
firmware_update.cpp: In member function ‘void DeRestPluginPrivate::checkFirmwareDevices()’:
firmware_update.cpp:548:73: error: ‘instance’ is not a member of ‘deCONZ::DeviceEnumerator’
     deCONZ::DeviceEnumerator *devEnumerator = deCONZ::DeviceEnumerator::instance();
                                                                         ^~~~~~~~
make[1]: *** [Makefile.Release:952: release/firmware_update.o] Fehler 1
make[1]: *** Es wird auf noch nicht beendete Prozesse gewartet....
make[1]: Verzeichnis „/home/pi/deconz-rest-plugin“ wird verlassen
make: *** [Makefile:40: release] Fehler 2

Beyond this, I'm sorry but I have no clue how to: 'You need to compile the code, replace the lib file, and re-incude the device (no need to delete the previous one)' Could you please advise?

But I will continue tomorrow evening. Bye for now 😴

Smanar commented 3 years ago

It s explained here https://github.com/dresden-elektronik/deconz-rest-plugin#install-deconz-development-package-optional-linux-only

To resume it s just 

sudo apt install deconz-dev
git clone --branch tuya_test5 https://github.com/Smanar/deconz-rest-plugin.git
cd deconz-rest-plugin
qmake && make -j2
sudo cp ../libde_rest_plugin.so /usr/share/deCONZ/plugins

And I think you have something not in date.

deconz-dev is already the newest version (2.12.00-raspbian-stretch-beta).

Or your OS

Pedder007 commented 3 years ago

Ah yes, I already used the 'manual' earlier

I checked the rest again: deconz-dev: 2.12.01-raspbian-buster-beta OS: Raspian Buster ID 10, just did an update/upgrade additionally - and there was 'something' open (dev system ... ;-) )

Then followed the instructions again (cloning your '...tuya_test5...' directory, mentioned above) and now I got no errors in the end :-) and also found another mistake, as I used the wrong port, see above: 8080, whereas 80 is the one of the original config.

Now I got back the following, but still no temp or humidity 😢 image

Pedder007 commented 3 years ago

There seems meanwhile beeing a second thread, also trying to capture the data form the device.

https://github.com/zigpy/zha-device-handlers/issues/862

But seems not to be easy, with that behaviour only to send temp- and humidity-data when the values crosses respective thresholds. Will follow both threads and still have the hope that they may find a solution ... 😄

github-actions[bot] commented 3 years ago

As there has not been any response in 21 days, this issue has been automatically marked as stale. At OP: Please either close this issue or keep it active It will be closed in 7 days if no further activity occurs.

github-actions[bot] commented 3 years ago

As there has not been any response in 28 days, this issue will be closed. @ OP: If this issue is solved post what fixed it for you. If it is not solved, request to get this opened again.

gmitch64 commented 2 years ago

I guess I didn't search properly before I opened a new issue - I probably only checked for open issues, my bad. Has there been any update with this device? I'm only seeing the Lux values coming back from the JSON, but looking at the other thread linked in this issue, there seems to have been some progress there

https://github.com/zigpy/zha-device-handlers/issues/862#issuecomment-1008422400

Is there any chance that the additional clusters can be added now?

G

Smanar commented 2 years ago

Lol, seriously, need to read thoses attributes and in the good order to unlock the device ?

github-actions[bot] commented 2 years ago

As there has not been any response in 21 days, this issue has been automatically marked as stale. At OP: Please either close this issue or keep it active It will be closed in 7 days if no further activity occurs.

github-actions[bot] commented 2 years ago

As there has not been any response in 28 days, this issue will be closed. @ OP: If this issue is solved post what fixed it for you. If it is not solved, request to get this opened again.

jaredgisin commented 1 year ago

This is still an issue with this device. Can I mail this device to a Dresden developer to figure this out? That would be way faster than this round-and-round for a year on GitHub. If someone is able to perform the analysis of this device to get it working let me know and I'll send you one.