Use TCP interface?

[dberlin]

Hey, so i notice this uses the UDP interface to the smart plugs, which has the fun issue of fragmentation (and it looks like you hit it on the forums).

The tp-link switches also respond on TCP port 9999, and there is now a raw TCP socket interface on hubitat that lets you control the sockets (and so you don't have to worry about multiple responses or fragmentation).

Any reason not to move over to using it?

(I'm happy to do porting work, just want to make sure i'm not missing something) Unfortunately, discovery is still UDP broadcast only, which hubitat can't really do.

[DaveGut]

Thanks for the offer. I have done some initial testing - but without success. Any assistance would be appreciated. I have placed a test driver at: "https://github.com/DaveGut/Hubitat-TP-Link-Integration/blob/master/tcpTest/tcpTest.groovy"

This driver

• runs the UDP version of the refresh - returning results as info messages.
• runs the TCP version - returns the results as warn messages (for ease of reading

Did not have success. The UDP worked; not the TCP. Possible causes:

• TCP comms miscoded
• Encryption not being done as I expected (see note in Method tcpEncrypt). This method is the same as the udpEncrypt except the header.

[dberlin]

I'll take a look. Just so you have a testbed, here's a CLI you can use that uses TCP (It's a modified version of a proof of concept someone else wrote): https://gist.github.com/dberlin/b980a27557e56d8eef81a6be55752ddc

[dberlin]

Note: The first 4 bytes of the TCP response are the length of the message (minus the 4 byte header), in network byte order. I wonder if your json parsing is failing because of trying to parse that

[DaveGut]

For the UDP, no. The Hubitat code decrypts the message and I have captured that message. If the length of the decrypted return is greater than 1023 characters, the return message truncates. I used to concatenate this with the second message (when I used the node.js server as a bridge) and then successfully parse.

What I am seeing in testing is there is nor return to the parse method (before doing any parsing). I think the encryption is not being done correctly or the comms are not going through.

[dberlin]

The comms are not going through, AFAICT.

There is one small bug in the TCP encryption: It should be six zeros, not 8 (AFAICT).

I modified the python code to print the hex string, and i get: python tplink_smartplug.py -t 192.168.1.14 -c info sending: 0000001dd0f281f88bff9af7d5ef94b6d1b4c09fec95e68fe187e8caf08bf68bf6

Modifying your code to use 6 zeros and print the encrypted string, i get: TCP sending message is 0000001dd0f281f88bff9af7d5ef94b6d1b4c09fec95e68fe187e8caf08bf68bf6

So they are identical.

Next step would be to pull out wireshark and see if it's really sending it.

On Mon, Jul 8, 2019 at 10:01 AM David Gutheinz notifications@github.com wrote:

For the UDP, no. The Hubitat code decrypts the message and I have captured that message. If the length of the decrypted return is greater than 1023 characters, the return message truncates. I used to concatenate this with the second message (when I used the node.js server as a bridge) and then successfully parse.

What I am seeing in testing is there is nor return to the parse method (before doing any parsing). I think the encryption is not being done correctly or the comms are not going through.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/DaveGut/Hubitat-TP-Link-Integration/issues/6?email_source=notifications&email_token=AACPI2746SI57SXHPKM2FO3P6NXE5A5CNFSM4H6V2QR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZNW44Q#issuecomment-509308530, or mute the thread https://github.com/notifications/unsubscribe-auth/AACPI2ZHXEUB2AKTLQ37EJTP6NXE5ANCNFSM4H6V2QRQ .

[DaveGut]

Thanks. I agree with your comments (my error in the code). There are still queries on Hubitat on actual format for the send message. Unfortunately, I do not have the wireshark setup for capturing Hubitat to Switch traffic (sigh). What I will try is direct the message to my PC running a node.js TCP server and see if that sees the message (I have done that in the past with minor success). Dav

[DaveGut]

Created a quickie TCP Node.Js server and tested on my PC. It works. Generated Command (from tcpEncrypt): 0000001dd0f281f88bff9af7d5ef94b6d1b4c09fec95e68fe187e8caf08bf68bf6

Command received at PC: A new connection has been established. Data received from client <Buffer 00 00 00 1d d0 f2 81 f8 8b ff 9a f7 d5 ef 94 b6 d1 b4 c0 9f ec 95 e6 8f e1 87 e8 ca f0 8b f6 8b f6> Closing connection with the client

Line-by-line compare (PC on top) 0000001dd0f281f88bff9af7d5ef94b6d1b4c09fec95e68fe187e8caf08bf68bf6 0000001dd0f281f88bff9af7d5ef94b6d1b4c09fec95e68fe187e8caf08bf68bf6 UDP : d0f281f88bff9af7d5ef94b6d1b4c09fec95e68fe187e8caf08bf68bf6

Node.js server:
test .js.txt

[DaveGut]

Success. I am getting and doing the initial processing (to known JSON input) using RAW TCP. Final code is below (for your records). Thanks for the not, push, and assistance.

Dave

tcpTest.groovy.txt

[DaveGut]

dberlin, Again thanks for the help. Issue: TP-Link has been busy disabling TCP communications on port 9999. Hubitat support both TCP Protocol.RAW_LAN messages and TCP rawSocket messages (differences in implementation / overhead). Bulbs: These support the TCP rawSocket messages. Do not support the TCP Protocol.RAW_LAN. Early Plugs and Switches: Support both TCP protocols. HS-107 Multi-Plug: These support the TCP rawSocket messages. Do not support the TCP Protocol.RAW_LAN. HS-300 Multi-Plug with EM: Does not support either TCP protocol.

[dberlin]

How are your determining this?

I have an hs300 and it works fine over tcp (I am using the latest firmware).

In fact, that is what the kasa app is doing on my phone - communicating with it over tcp.

It only issues cloud commands when I am not on local network.

On Tue, Jul 9, 2019, 7:22 AM David Gutheinz notifications@github.com wrote:

dberlin, Again thanks for the help. Issue: TP-Link has been busy disabling TCP communications on port 9999. Hubitat support both TCP Protocol.RAW_LAN messages and TCP rawSocket messages (differences in implementation / overhead). Bulbs: These support the TCP rawSocket messages. Do not support the TCP Protocol.RAW_LAN. Early Plugs and Switches: Support both TCP protocols. HS-107 Multi-Plug: These support the TCP rawSocket messages. Do not support the TCP Protocol.RAW_LAN. HS-300 Multi-Plug with EM: Does not support either TCP protocol.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/DaveGut/Hubitat-TP-Link-Integration/issues/6?email_source=notifications&email_token=AACPI267XEFIHTQ3YGFBKA3P6SNK5A5CNFSM4H6V2QR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZQNQZQ#issuecomment-509663334, or mute the thread https://github.com/notifications/unsubscribe-auth/AACPI24DSXPVRBYS3TSWPCDP6SNK5ANCNFSM4H6V2QRQ .

[DaveGut]

If you have Hubitat, you can test yourself. Attached is the driver for a manual installation Create the virtual device using the below driver. Enter the IP address of any plug/switch other than HS-300. Turn on logging Run Refresh (it will run UDP, raw TCP, and RAW_LAN) Expected result for a HS-115: successful return from each method. Enter the IP address of your HS-300. Run the refresh again Note there is NO response from the TCP protocols.

I have attached the driver and a copy of the log files for the HS-105 and HS-300.

HS-300 Return.txt HS-105 Return.txt

TEST DRIVER.groovy.txt

Dave

[DaveGut]

That of course does not mean that the problem is not with Hubitat. I did note (physical observation) that on the raw TCP socket, the device turned on or off (if sent the command) - but did not respond. Not true for the RAW_LAN protocol suggested by the Hubitat staff.

Do you perhaps know the port that the Kasa App uses (it may be 1040 or some other port).

[dberlin]

I'll take a look. I do know if i send the command to my hs300 manually from the command line, it responds fine over TCP.

Sent: {"system":{"set_relay_state":{"state":1}}} Received: {"system":{"set_relay_state":{"err_code":0}}}

On Tue, Jul 9, 2019 at 8:33 AM David Gutheinz notifications@github.com wrote:

That of course does not mean that the problem is not with Hubitat. I did note (physical observation) that on the raw TCP socket, the device turned on or off (if sent the command) - but did not respond. Not true for the RAW_LAN protocol suggested by the Hubitat staff.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/DaveGut/Hubitat-TP-Link-Integration/issues/6?email_source=notifications&email_token=AACPI2ZQ5D4UNZUYJO6VAZLP6SVTNA5CNFSM4H6V2QR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZQUU3A#issuecomment-509692524, or mute the thread https://github.com/notifications/unsubscribe-auth/AACPI25R5S4W43A5N6COJG3P6SVTNANCNFSM4H6V2QRQ .

[DaveGut]

I fully understand. However, I am constrained by using the Hubitat raw TCP and / or Protocol.RAW_LAN (TCP) implementations. If they do not work for some of the devices, it is of no help.

From: Daniel Berlin notifications@github.com Sent: Tuesday, July 9, 2019 10:49 To: DaveGut/Hubitat-TP-Link-Integration Hubitat-TP-Link-Integration@noreply.github.com Cc: David Gutheinz djgutheinz@outlook.com; State change state_change@noreply.github.com Subject: Re: [DaveGut/Hubitat-TP-Link-Integration] Use TCP interface? (#6)

I'll take a look. I do know if i send the command to my hs300 manually from the command line, it responds fine over TCP.

Sent: {"system":{"set_relay_state":{"state":1}}} Received: {"system":{"set_relay_state":{"err_code":0}}}

On Tue, Jul 9, 2019 at 8:33 AM David Gutheinz notifications@github.com wrote:

That of course does not mean that the problem is not with Hubitat. I did note (physical observation) that on the raw TCP socket, the device turned on or off (if sent the command) - but did not respond. Not true for the RAW_LAN protocol suggested by the Hubitat staff.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/DaveGut/Hubitat-TP-Link-Integration/issues/6?email_source=notifications&email_token=AACPI2ZQ5D4UNZUYJO6VAZLP6SVTNA5CNFSM4H6V2QR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZQUU3A#issuecomment-509692524, or mute the thread https://github.com/notifications/unsubscribe-auth/AACPI25R5S4W43A5N6COJG3P6SVTNANCNFSM4H6V2QRQ .

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHubhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FDaveGut%2FHubitat-TP-Link-Integration%2Fissues%2F6%3Femail_source%3Dnotifications%26email_token%3DAGAI4M2QO5YMB3TW3PYWGQLP6SXPPA5CNFSM4H6V2QR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZQWGZA%23issuecomment-509698916&data=02%7C01%7C%7Cc2fed4743da34d88ebed08d70484fca8%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636982841522896849&sdata=5xBIzjxaI4RbvAxvA%2FUwBhuK15IHwiqiJPyWR%2FUzl18%3D&reserved=0, or mute the threadhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAGAI4M6KS24O4HDQSEJAZ3TP6SXPPANCNFSM4H6V2QRQ&data=02%7C01%7C%7Cc2fed4743da34d88ebed08d70484fca8%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636982841522916865&sdata=FIIW%2Fi8CQkwUX%2F0DbghXCNAX7EwEy3n%2FkpRTdpsIpiY%3D&reserved=0.

[dberlin]

Yup, i get it. If that ends up being the case, i'll file bugs with hubitat so they know :)

On Tue, Jul 9, 2019 at 8:52 AM David Gutheinz notifications@github.com wrote:

I fully understand. However, I am constrained by using the Hubitat raw TCP and / or Protocol.RAW_LAN (TCP) implementations. If they do not work for some of the devices, it is of no help.

From: Daniel Berlin notifications@github.com Sent: Tuesday, July 9, 2019 10:49 To: DaveGut/Hubitat-TP-Link-Integration < Hubitat-TP-Link-Integration@noreply.github.com> Cc: David Gutheinz djgutheinz@outlook.com; State change < state_change@noreply.github.com> Subject: Re: [DaveGut/Hubitat-TP-Link-Integration] Use TCP interface? (#6)

I'll take a look. I do know if i send the command to my hs300 manually from the command line, it responds fine over TCP.

Sent: {"system":{"set_relay_state":{"state":1}}} Received: {"system":{"set_relay_state":{"err_code":0}}}

On Tue, Jul 9, 2019 at 8:33 AM David Gutheinz notifications@github.com wrote:

That of course does not mean that the problem is not with Hubitat. I did note (physical observation) that on the raw TCP socket, the device turned on or off (if sent the command) - but did not respond. Not true for the RAW_LAN protocol suggested by the Hubitat staff.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub < https://github.com/DaveGut/Hubitat-TP-Link-Integration/issues/6?email_source=notifications&email_token=AACPI2ZQ5D4UNZUYJO6VAZLP6SVTNA5CNFSM4H6V2QR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZQUU3A#issuecomment-509692524>,

or mute the thread < https://github.com/notifications/unsubscribe-auth/AACPI25R5S4W43A5N6COJG3P6SVTNANCNFSM4H6V2QRQ>

.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub< https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FDaveGut%2FHubitat-TP-Link-Integration%2Fissues%2F6%3Femail_source%3Dnotifications%26email_token%3DAGAI4M2QO5YMB3TW3PYWGQLP6SXPPA5CNFSM4H6V2QR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZQWGZA%23issuecomment-509698916&data=02%7C01%7C%7Cc2fed4743da34d88ebed08d70484fca8%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636982841522896849&sdata=5xBIzjxaI4RbvAxvA%2FUwBhuK15IHwiqiJPyWR%2FUzl18%3D&reserved=0>, or mute the thread< https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAGAI4M6KS24O4HDQSEJAZ3TP6SXPPANCNFSM4H6V2QRQ&data=02%7C01%7C%7Cc2fed4743da34d88ebed08d70484fca8%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636982841522916865&sdata=FIIW%2Fi8CQkwUX%2F0DbghXCNAX7EwEy3n%2FkpRTdpsIpiY%3D&reserved=0>.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/DaveGut/Hubitat-TP-Link-Integration/issues/6?email_source=notifications&email_token=AACPI23EIUXK5PAGJMRGVTTP6SX3TA5CNFSM4H6V2QR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZQWQ3Q#issuecomment-509700206, or mute the thread https://github.com/notifications/unsubscribe-auth/AACPI23M3WXBSC5Y3MAUQLLP6SX3TANCNFSM4H6V2QRQ .

[DaveGut]

Did some further testing. The Raw_Tcp commands return correct data for the three Energy Monitor queries. It does not work for refresh on the HS-300, but does work for refresh on the LB120/130. I could not test on the HS-110 (no device to test). Result: I will test again on Jul 31 and see if the returns are fragmented. If not fragmented, I will use the Raw_TCP to get Energy Monitor data (and modify code to remove work-around). If fragmented, no changes. Experimentation indicates that the problem is in the devices, not the Hubitat commands. go figure. The new test driver is below, if you have any EM devices.

Dave TEST DRIVER.groovy.txt