rgerganov / py-air-control

Command line app for controlling Philips air purifiers
MIT License
260 stars 52 forks source link

Unable to get CoAP to work with AC2889/60 #35

Closed ryancdotorg closed 4 years ago

ryancdotorg commented 4 years ago

Firmware (from the app) reports as

Firmware Version: AWS_Philips_AIR@54.2 Device Version: 1.0.7 Model: AC2889/10

Any attempt to send a command just gets back {"status":"failed"} (looking at it via tcpdump). Also from tcpdump, the app appears to be exclusively using the cloud based system to talk to it. If I put both on a wifi network not connected to the internet, even the app doesn't work.

Cyber1000 commented 4 years ago

Information on progress

Setting of values works now 🎈 (updated https://github.com/Cyber1000/py-air-control/tree/coap_1_0_7)!

airctrl --ipaddr [ip] --version 1.0.7 --mode M --om 2
airctrl --ipaddr [ip] --version 1.0.7 --pwr 0

More or less whatever is stated on the mainpage: https://github.com/rgerganov/py-air-control I couldn't test all functions since every device has different functions and my AC2889 has no humidity for example. I assume they will work though.

What won't work: Wi-Fi setup, --firmware, --filters


Questions, Annotations, ...

At everyone who got the execution of commands working (@timbuktu-t @holomekc): Between two commands I sometimes have to wait some seconds, otherwise my device doesn't accept it. Most time I had this problem with mode-switching, for example:

airctrl --ipaddr [ip] --version 1.0.7 --mode M --om 2
wait 10s # not waiting here results in response "failed" for the following command
airctrl --ipaddr [ip] --version 1.0.7 --mode A
wait 10s
airctrl --ipaddr [ip] --version 1.0.7 --mode M --om 2

Did you experience something similar? maybe it was just my smartphone connected at the same time, that interferred here ...

Another interesting point to everyone who implements this in other languages (@timbuktu-t @holomekc ): On setting commands, the deviceId (besides EnduserId) isn't necessary, this will do as well: {"state":{"desired":{"CommandType":"app","DeviceId":"","EnduserId":"", key: value }}}

@timbuktu-t Thanks for the information about the observe, I missed that out. Anyway my point was more or less the version "0.2.1" in the logs and @szogi saying the device updated to 1.0.7. I wasn't expecting version 0.2.1 to work with my or your code but with another (older) coap-code implemented by @shexbeer.

@shexbeer would you try my branch with airctrl --ipaddr [ip] --version 1.0.7 ? Since you have version 0.2.1 I'm interested, if my code works here too ...

@bobzomer and @szogi thanks for the encrypted payload and @rgerganov thanks for finding out. I didn't have time to think about how to test this on my side efficiently. Maybe I find out something tomorrow. I've taken the coap-part mostly from @shexbeer, no real experience here.

@holomekc code and therefore possible different implementations are always welcome, maybe I've missed some edgecases. You may drop a note that you can't mantain it and/or remove the issues section on your repo, that may help. I would be interested in the code ...

laurence-syree commented 4 years ago

Confirming that this version is able to set values as you say on my AC2889/10 and as you say sometimes it fails if commands sent to quickly in succession however I also noted that it sometimes fails the first time I attempt to sent a command and succeeds on subsequent attempts. My smartphone was also connected at the same time during testing.

Thanks for the great work as ever.

holomekc commented 4 years ago

@Cyber1000 Hi I do not have the issue that commands executed without any pause fails. I tested it witth switch between auto and manual mode. But there seems to be some actions which fail. E.g. Switch light off. aqil 0 does not work for me. uil false also fails. But aqil 1-100 works fine. My theory i need to set aqil 0 and uil false in one message.

Give me minutes to wake up and i will upload the code

holomekc commented 4 years ago

Here we go: https://github.com/holomekc/pch-code-collection

michalboronski commented 4 years ago

Quick question : do newer models that are using CoAP still announce themselves through Upnp?

timbuktu-t commented 4 years ago

@michalboronski On my router I can see udp packages on port 5353 going to my AC3829/10, so there´s a slim chance it uses a derivate of mDNS. I don´t see any response packages though, and it does not show up in Network Analyzer for UPnP or Bonjour.

Cyber1000 commented 4 years ago

Updated to a new version, but it's not much added function, just a little cleanup and smaller fixes. @szogi's string is 1138, @bobzomer 's 1136 bytes big, not including part of data - mine only 1128 (inlcuding all data including a 64byte digest for verification at the end)

It seems that COAP "SHOULD fit within a single IP packet" (https://tools.ietf.org/html/rfc7252#section-4.6http://) and that you can make blockwise transfers to get multiple messages. Unfortunately I haven't found out how to change my code to do this, perhaps @shexbeer (or someone else) has a deeper knowledge of coap. CoAPthon (the library included here) has a blockwise-feature, but I don't know if that's the solution (and how to implement and test it). Any ideas are welcome


So what's missing for the PR

What I won't do for now (but maybe near future)

AndiH commented 4 years ago

@Cyber1000 Just to mention this: Your fork also works for my AC2889/10! Thanks!

➜ pipx install git+https://github.com/Cyber1000/py-air-control.git@coap_1_0_7
…
➜ airctrl --version 1.0.7 --ipaddr 192.168.1.49
[name]: Schlafzimmer
[type]: AC2889
[modelid]: AC2889/10
[swversion]: 1.0.7
[om]: 1
[pwr]: 1
…
bocsitomcsi commented 4 years ago

@Cyber1000 first of all thanks for your hard work. Finally I managed to configure my AC2729/50:

bash-5.0# airctrl --ipaddr 192.168.0.54 --version 1.0.7 --mode A -d
From ('192.168.0.54', 5683), To None, NON-3223, CONTENT-Pl, [Content-Type: 50, ] {"status":"success"}...20 bytes
bash-5.0# airctrl --ipaddr 192.168.0.54 --version 1.0.7 --pwr 0 -d
From ('192.168.0.54', 5683), To None, NON-48404, CONTENT-pz, [Content-Type: 50, ] {"status":"success"}...20 bytes
bash-5.0# airctrl --ipaddr 192.168.0.54 --version 1.0.7 --pwr 1 -d
From ('192.168.0.54', 5683), To None, NON-13488, CONTENT-gd, [Content-Type: 50, ] {"status":"success"}...20 bytes

However when I would like to print out the basic information I got the following:

bash-5.0# airctrl --ipaddr 192.168.0.54 --version 1.0.7 -d
Message from device got corrupted
Raw status: {}

Is it a know limitation?

Cyber1000 commented 4 years ago

@bocsitomcsi You seem to have the seem problem like https://github.com/rgerganov/py-air-control/issues/35#issuecomment-625288284 (btw. it's AC2729/50 too) and some others here. The returning message is too long for a single coap-package, and is cut somewhere in the middle. I don't know so far how to get multiple packages.

I just catch the exception now and you get a "Message from device got corrupted" therefore.

I'll make a PR within the next 2 days with this known limitation and will look into this in the near future.

bocsitomcsi commented 4 years ago

Thanks @Cyber1000 for the clarification.

daenney commented 4 years ago

@michalboronski

Quick question : do newer models that are using CoAP still announce themselves through Upnp?

What I've seen so far is multicast CoAP for /sys/dev/info as the discovery mechanism. I've implemented that in my own (Go) library: https://github.com/hemtjanst/klimat/blob/master/cmd/klimat/discover/discover.go. This works reliably for me to find the device on the network.

Cyber1000 commented 4 years ago

I've created this PR, it has some limitations, but I think they should better get an own PR, since that will need some more time to investigate

rgerganov commented 4 years ago

I have merged @Cyber1000's PR into master, you are welcome to test with HEAD. I will release a new PyPI version when we fix the issues with the truncated CoAP response.

Thanks to everyone who contributed to the project so far!

bobzomer commented 4 years ago

When we look at a Wireshark capture, we can see that the packet is not truncated. But the packet received by the library is indeed truncated.

The problem comes from the CoAPthon3 library. We can see in coapthon/client/coap.py, line 239, method receive_datagram:

                datagram, addr = self._socket.recvfrom(1152)

I've cloned the CoAPthon3 library and modified this line to:

                datagram, addr = self._socket.recvfrom(1500)

Now it works like a charm!

I will try submitting a PR to this external dependency, but there has been no update to the pip package since 2018, even if several PRs have been accepted since...

bobzomer commented 4 years ago

Some of the retrieved is not interpreted by the code:

[name]                        Name: Salon
[type]                        Type: AC3039
[modelid]                     ModelId: AC3039/10
[swversion]                   Version: Ms3104
[language]                    language: EN
[DeviceVersion]               DeviceVersion: 1.0.4
[om]                          Fan speed: 1
[pwr]                         Power: ON
[cl]                          Child lock: False
[aqil]                        Light brightness: 100
[uil]                         Buttons light: ON
[uaset]                       uaset: A
[mode]                        Mode: AG
[pm25]                        PM25: 6
[iaql]                        Allergen index: 2
[aqit]                        Air quality notification threshold: 4
[tvoc]                        tvoc: 1
[ddp]                         Used index: PM2.5
[rddp]                        rddp: 1
[fltt1]                       HEPA filter type: NanoProtect Filter Series 3 (FY2422)
[fltt2]                       Active carbon filter type: none
[fltsts0]                     Pre-filter and Wick: clean in 294 hours
[fltsts1]                     HEPA filter: replace in 4704 hours
[fltsts2]                     Active carbon filter: replace in 65535 hours
[filna]                       filna: 0
[filid]                       filid: 0
[ota]                         Over the air updates: ck
[Runtime]                     Runtime: 20.79 hours
[WifiVersion]                 WifiVersion: AWS_Philips_AIR@54.2
[ProductId]                   ProductId: 660c6902649b11e9a1e3061302926720
[DeviceId]                    DeviceId: 0b529c58af8111e9a1e3061302926720
[StatusType]                  StatusType: localcontrol
[ConnectType]                 ConnectType: Localcontrol

I don't know for all, but at least tvoc seems to be the VOC content index (1 is good, 4 is vera bad).

wico commented 4 years ago

How awesome! I can confirm that it now also works with a AC4236/10 after applying https://github.com/rgerganov/py-air-control/issues/35#issuecomment-631899965.

[name]                        Name: Loft
[type]                        Type: AC4236
[modelid]                     ModelId: AC4236/10
[swversion]                   Version: Ms4106
[language]                    language: EN
[DeviceVersion]               DeviceVersion: 1.0.6
[om]                          Fan speed: 1
[pwr]                         Power: ON
[cl]                          Child lock: False
[aqil]                        Light brightness: 100
[uil]                         Buttons light: ON
[uaset]                       uaset: A
[mode]                        Mode: AG
[pm25]                        PM25: 4
[iaql]                        Allergen index: 1
[aqit]                        Air quality notification threshold: 10
[tvoc]                        tvoc: 1
[ddp]                         Used index: IAI
[rddp]                        rddp: 0
[fltt1]                       HEPA filter type: NanoProtect Filter Series 3 (FY2422)
[fltt2]                       Active carbon filter type: none
[fltsts0]                     Pre-filter and Wick: clean in 224 hours
[fltsts1]                     HEPA filter: replace in 4800 hours
[fltsts2]                     Active carbon filter: replace in 65535 hours
[filna]                       filna: AC3036
[filid]                       filid: AC3036[...]
[ota]                         Over the air updates: ck
[Runtime]                     Runtime: 2.77 hours
[WifiVersion]                 WifiVersion: AWS_Philips_AIR@54.2
[ProductId]                   ProductId: [...]
[DeviceId]                    DeviceId: [...]
[StatusType]                  StatusType: localcontrol
[ConnectType]                 ConnectType: Localcontrol
Cyber1000 commented 4 years ago

@bobzomer nice catch 👍! I've also had some questions to the coapthon3 repo owner: https://github.com/Tanganelli/CoAPthon3/issues/28 The "request.observe = 0" seems to be necessary and it doesn't seem to have to do with multiple blocks (that was my first idea), since you are getting all in one message from the device.

So hope we are getting a pip soon

Some of the retrieved is not interpreted by the code

I've taken what was there from the original implementation of http and coap and what I've found on my device. Your device seems to have a lot more information and even a mode AG, which isn't translated. You may extend the statusTransformer, when you have the information or perhaps someone knows what the additional values stand for. image

Cyber1000 commented 4 years ago

From your output, missing are:

[language]                    language: EN
[uaset]                       uaset: A
[mode]                        Mode: AG
[tvoc]                        tvoc: 1
[rddp]                        rddp: 1
[filna]                       filna: 0
[filid]                       filid: 0
bobzomer commented 4 years ago
wico commented 4 years ago

Regarding "AG": This is shown when I enable the "Automatic" mode on my device (the post-2019 purifiers seems to have only 3 modes (Turbo, Automatic, Sleep)).

Also, in my case, I answered the "purpose questions" with:

Not sure if it makes any difference, but I basically clicked almost all options and the result is "AG". :)

spider7611 commented 4 years ago

After this modification:

The problem comes from the CoAPthon3 library. We can see in coapthon/client/coap.py, line 239, method receive_datagram:

                datagram, addr = self._socket.recvfrom(1152)
I've cloned the CoAPthon3 library and modified this line to:

                datagram, addr = self._socket.recvfrom(1500)

Working my model Id: AC3858/50 too! airctrl --ipaddr 192.168.1.102 --protocol coap --pwr 1 Need home assistant integration! :)

[name] Name: Living Room [type] Type: AC3858 [modelid] ModelId: AC3858/50 [swversion] Version: Ms4102 [language] language: EN [om] Fan speed: 1 [pwr] Power: ON [cl] Child lock: False [aqil] Light brightness: 0 [uil] Buttons light: OFF [uaset] uaset: A [mode] Mode: AG [pm25] PM25: 3 [iaql] Allergen index: 1 [aqit] Air quality notification threshold: 10 [tvoc] tvoc: 1 [ddp] Used index: PM2.5 [rddp] rddp: 1 [fltt1] HEPA filter type: NanoProtect Filter Series 3 (FY2422) [fltt2] Active carbon filter type: none [fltsts0] Pre-filter and Wick: clean in 155 hours [fltsts1] HEPA filter: replace in 4560 hours [fltsts2] Active carbon filter: replace in 65535 hours [filna] filna: AC3036 [filid] filid: AC3036.... [ota] Over the air updates: no [Runtime] Runtime: 0.96 hours [WifiVersion] WifiVersion: AWS_Philips_AIR@53 [ProductId] ProductId: ..... [DeviceId] DeviceId: .... [StatusType] StatusType: localcontrol [ConnectType] ConnectType: Localcontrol

rgerganov commented 4 years ago

@spider7611 thanks for the feedback

I see they merged the patch submitted by @bobzomer for this: https://github.com/Tanganelli/CoAPthon3/issues/29

maybe we can put dependency to the latest code on github until they release on PyPI.

Thoughts?

GSzabados commented 4 years ago

@spider7611 thanks for the feedback

I see they merged the patch submitted by @bobzomer for this: Tanganelli/CoAPthon3#29

maybe we can put dependency to the latest code on github until they release on PyPI.

Thoughts?

@rgerganov The comment regarding the bug in the README.md should be updated, as it says pip instead of pip3, and it does get the wrong package.

PS.: Added a PR for that. It caused me a bit of a headache until I found out that the coal.py hasn't been changed with the aforementioned 1500, but installed an old version.

rgerganov commented 4 years ago

PS.: Added a PR for that. It caused me a bit of a headache until I found out that the coal.py hasn't been changed with the aforementioned 1500, but installed an old version.

Sorry about the misleading README and thank you for the PR. See my comment there.

Cyber1000 commented 4 years ago

Maybe something like this would also work to include a package from a github commit:

Spoiler:

But maybe anyone of you finds a way or can use this information.

rgerganov commented 4 years ago

I also spend some time looking into this and came to the conclusion that putting a dependency to source control in setup.py is deprecated and considered bad practice. This is what the pip documentation says about cases like ours:

Requirements files are used to override a dependency with a local patch that lives in version control. For example, suppose a dependency SomeDependency from PyPI has a bug, and you can’t wait for an upstream fix. You could clone/copy the src, make the fix, and place it in VCS with the tag sometag. You’d reference it in your requirements file with a line like so:

git+https://myvcs.com/some_dependency@sometag#egg=SomeDependency

So I updated the requirements file and put a note in the README (which needs to be fixed :smiley: ).

GSzabados commented 4 years ago

@rgerganov, @Cyber1000 thanks for the fixes implemented above. It is working with my AC3829/50.

Only question, does the coap protocol requires to be run with sudo? Once I tried to run without sudo it gives a "Message from device got corrupted" result. Is it requires due to the /sys/dev/sync ?

Cyber1000 commented 4 years ago

I just tried it out:

My best guess is that you have installed CoAPthon3 multiple times:

I would try as "normal" user:

sudo pip3 uninstall CoApthon3
pip3 uninstall CoApthon3
pip3 install -U git+https://github.com/Tanganelli/CoAPthon3@89d5173

Maybe you have also installed some version with pip (and not pip3) as you mentioned correctly, that the README was wrong. Then a uninstall command with pip, similar to the shown above might help.

GSzabados commented 4 years ago

@Cyber1000, ok thanks! That makes sense. And you have just pointed out that my /home/pi/.local/bin was missing somehow from PATH. Thanks for that! All works now correctly.

szogi commented 4 years ago

Guys, nice work, thank you!

# airctrl --ipaddr 172.16.0.140 --protocol coap
[name]                        Name: Bedroom
[type]                        Type: AC2729
[modelid]                     ModelId: AC2729/50
[swversion]                   Version: 0.2.1
[om]                          Fan speed: silent
[pwr]                         Power: ON
[cl]                          Child lock: False
[aqil]                        Light brightness: 0
[uil]                         Buttons light: OFF
[mode]                        Mode: sleep
[func]                        Function: Purification
[rhset]                       Target humidity: 50
[rh]                          Humidity: 59
[temp]                        Temperature: 26
[pm25]                        PM25: 13
[iaql]                        Allergen index: 4
[aqit]                        Air quality notification threshold: 10
[ddp]                         Used index: PM2.5
[rddp]                        rddp: 1
[err]                         [ERROR] Message: 49495
[wl]                          Water level: 0
[fltt1]                       HEPA filter type: NanoProtect Filter Series 3 (FY2422)
[fltt2]                       Active carbon filter type: NanoProtect Filter AC (FY2420)
[fltsts0]                     Pre-filter and Wick: clean in 0 hours
[fltsts1]                     HEPA filter: replace in 58 hours
[fltsts2]                     Active carbon filter: replace in 58 hours
[wicksts]                     Wick filter: replace in 58 hours
[ota]                         Over the air updates: ck
[Runtime]                     Runtime: 0.02 hours
[WifiVersion]                 WifiVersion: AWS_Philips_AIR@56.4
[ProductId]                   ProductId: xxxxxxxxxxxxx
[DeviceId]                    DeviceId: xxxxxxxxxxxxxxxxx
[StatusType]                  StatusType: localcontrol
[ConnectType]                 ConnectType: Localcontrol
rgerganov commented 4 years ago

Thank @szogi for the feedback :) Also thank everyone who contributed for this work! I am closing this now.