Closed nao-pon closed 2 years ago
In the old days all the UDP functionality was very inefficient, and did not play nicely with Asyncio event loops at all. Basically I would fire a packet, listen for a response (which involved waiting for 3 seconds and pausing the entire Home Assistant) and then process the packet. It was fine when it was just me and my solo air conditioner but as more people started using the code with multiple devices, what was happening was that it was bogging Home Assistant down listening for messages.
As more people started using the Home Assistant component; and with multiple devices, I needed a better way of processing the packets in a manner that was asyncio friendly, and would work with Home Assistants event loop. To that end I used https://github.com/bashkirtsevich-llc/aioudp which provided a ready to use UDP service that was asyncio friendly. I rewrote the code in about a day.
What aio-udp-server
allowed me to do was decouple the transmit and receive functionality, the receive is basically a listener that keeps the socket open at all times and can process any incoming packet that lands on port 3610. The transmit functionality works as UDP is expected to work, which is fire the message and forget. I use the TID to try to have some idea of whether or not packets are getting lost or are being received.
The drama we have is two fold:
To fix the first problem, we would probably have to rewrite aio-udp-server
to enable a multicast listener (which should be possible, but would take some time). It would probably make sense to move aio-udp-server
into pychonet and rewrite it as its not a particularly large library...
To fix the second problem we would have to update the listener logic to process any status change property announcements that are received on the multicast address.
Additionally, Home Assistant may or may not be able to listen to multicast; and that will really come down to the installation - if it's running in a Docker container for example it might not be possible to receive multicast packets (not that I am 100% sure).
There is some bare metal python3 socket code here for listening to multicast that I stole from stackoverflow:
import socket
import struct
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(('', 4242))
mreq = struct.pack("=4sl", socket.inet_aton("224.51.105.104"), socket.INADDR_ANY)
sock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq)
while True:
print sock.recv(10240)
Okay I think I have the listener subscribed to the multicast address. Ive incorporated the aio-udp-server code directly into the library.
Download latest and run in interactive mode:
python3 -m asyncio
from pychonet.lib.udpserver import UDPServer
from pychonet import Factory
from pychonet import ECHONETAPIClient as api
from pychonet import HomeAirConditioner
from pychonet import EchonetInstance
udp = UDPServer()
loop = asyncio.get_event_loop()
udp.run("0.0.0.0", 3610, loop)
server = api(udp)
server._debug_flag = True
Now, in Linux with Wireshark I could definitely see multicast packets being processed but there is a bit of work to do around the logic. MACOS didn't work because multicast is blocked.
You could try to add your device eg:
await server.discover('192.168.1.6')
and then see if you can see multicast packets arriving from that device.
@scottyphillips Great! Very fast work!
I went into the shell of the homeassistant docker instance of the HomeAssistant OS and tried it.
I got the following message! (Some excerpts)
Unknown host - probably multicast message
Unknown host - probably multicast message
Unknown host - probably multicast message
Unknown host - probably multicast message
Unknown host - probably multicast message
Unknown host - probably multicast message
Unknown host - probably multicast message
Unknown host - probably multicast message
b'\x10\x81\t\x9b\x02r\x01\x05\xff\x01r\x03\xd5\x01\x0c\xee\x02\x00\xc8\xef\x01C'
('192.168.1.156', 2524)
Echonet Message Received - Processed data is {'EHD1': 16, 'EHD2': 129, 'TID': 2459, 'SEOJGC': 2, 'SEOJCC': 114, 'SEOJCI': 1, 'DEOJGC': 5, 'DEOJCC': 255, 'DEOJCI': 1, 'ESV': 114, 'OPC': [{'EPC': 213, 'PDC': 1, 'EDT': b'\x0c'}, {'EPC': 238, 'PDC': 2, 'EDT': b'\x00\xc8'}, {'EPC': 239, 'PDC': 1, 'EDT': b'C'}]}
Unknown host - probably multicast message
Unknown host - probably multicast message
Unknown host - probably multicast message
b'\x10\x81\t\x9c\x02r\x01\x05\xff\x01r\n\x80\x010\x90\x01B\xd0\x01B\x91\x02\x15\x1e\xd1\x01+\xe1\x01)\xe2\x01B\xe3\x01A\xd4\x01\x06\xe4\x01B'
('192.168.1.156', 2525)
Echonet Message Received - Processed data is {'EHD1': 16, 'EHD2': 129, 'TID': 2460, 'SEOJGC': 2, 'SEOJCC': 114, 'SEOJCI': 1, 'DEOJGC': 5, 'DEOJCC': 255, 'DEOJCI': 1, 'ESV': 114, 'OPC': [{'EPC': 128, 'PDC': 1, 'EDT': b'0'}, {'EPC': 144, 'PDC': 1, 'EDT': b'B'}, {'EPC': 208, 'PDC': 1, 'EDT': b'B'}, {'EPC': 145, 'PDC': 2, 'EDT': b'\x15\x1e'}, {'EPC': 209, 'PDC': 1, 'EDT': b'+'}, {'EPC': 225, 'PDC': 1, 'EDT': b')'}, {'EPC': 226, 'PDC': 1, 'EDT': b'B'}, {'EPC': 227, 'PDC': 1, 'EDT': b'A'}, {'EPC': 212, 'PDC': 1, 'EDT': b'\x06'}, {'EPC': 228, 'PDC': 1, 'EDT': b'B'}]}
Unknown host - probably multicast message
Unknown host - probably multicast message
Unknown host - probably multicast message
Unknown host - probably multicast message
Unknown host - probably multicast message
Unknown host - probably multicast message
Unknown host - probably multicast message
Unknown host - probably multicast message
Unknown host - probably multicast message
Unknown host - probably multicast message
b'\x10\x81\t\x9d\x02r\x01\x05\xff\x01r\x03\xd5\x01\x0c\xee\x02\x00\xc8\xef\x01C'
('192.168.1.156', 2526)
Echonet Message Received - Processed data is {'EHD1': 16, 'EHD2': 129, 'TID': 2461, 'SEOJGC': 2, 'SEOJCC': 114, 'SEOJCI': 1, 'DEOJGC': 5, 'DEOJCC': 255, 'DEOJCI': 1, 'ESV': 114, 'OPC': [{'EPC': 213, 'PDC': 1, 'EDT': b'\x0c'}, {'EPC': 238, 'PDC': 2, 'EDT': b'\x00\xc8'}, {'EPC': 239, 'PDC': 1, 'EDT': b'C'}]}
Unknown host - probably multicast message
Unknown host - probably multicast message
Unknown host - probably multicast message
P.S. One thing I was curious about was that the ECHONET Lite entity in the HA UI was disabled while I was experimenting with running python3 in the shell.
When I stopped python3 in the shell, it returned to normal again.
If this condition is intended or expected, then there is no problem.
It’s likely that the multicast listener is working then. All the repeated ‘unknown host’ messages are just the EPC processing code iterating over multiple EPCs (was tired last night and just wanted to get a multicast proof of concept out the door)
the disabling of the echonetlite node is (I speculate) probably due to hijacking the UDP port
The code in question if you want to play around is: echonetMessageReceived(self, raw_data, addr) in echonetapiclient.py
We can probably just treat the packet no differently to any other received echonet message but just populate the missing host details in the main state dict.
Im really limited in my ability to test multicast echonet lite outside of firing a manually crafted discovery packet - MOEKADEN room doesn't fire them - so ill have to let you take the lead on this one...
I tried the latest version of the test.
However, the echonetlite integration did not boot up properly.
Is there anything else I need to fix?
EDITED:
Reverting to https://github.com/scottyphillips/pychonet/commit/9300dc5ce4575f32902d0caba3435e1d3529b2e2 and back to "from aioudp import UDPServer" works fine.
Probably. Home assistant log should tell you what’s broken. I wouldn’t be using the latest pychonet with home assistant yet. Its not configured as a pip release. I would be running independently where you can monitor echonet packets in real time.
Get Outlook for iOShttps://aka.ms/o0ukef
From: Naoki Sawada @.> Sent: Sunday, June 19, 2022 5:36:40 PM To: scottyphillips/pychonet @.> Cc: Scott Phillips @.>; Mention @.> Subject: Re: [scottyphillips/pychonet] Listen to UDP port 3610 (Issue #40)
I tried the latest version of the test.
However, the echonetlite integration did not boot up properly.
Is there anything else I need to fix?
— Reply to this email directly, view it on GitHubhttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fscottyphillips%2Fpychonet%2Fissues%2F40%23issuecomment-1159641592&data=05%7C01%7C%7C9706be35cec4490ede9508da51c67348%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637912210040723775%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=fAs88rQTHhOjbjf3C2DI4EdxZbnNeNcWBI0ALife3NM%3D&reserved=0, or unsubscribehttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAGHQNDGGZSSN5AHEHPWSKK3VP3EYRANCNFSM5ZBPRULA&data=05%7C01%7C%7C9706be35cec4490ede9508da51c67348%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637912210040723775%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=OrqpDQBNRFRtMPx%2BqNeCo1zMd2abusqWVsqso1DkIAw%3D&reserved=0. You are receiving this because you were mentioned.Message ID: @.***>
Its probably easy enough to fix - change line 90 in _init__.py in echonetlite_homeassistant
From this:
server = ECHONETAPIClient(server=udp, loop=loop)
To this:
server = ECHONETAPIClient(udp)
When I was reviewing the code I got rid of some dud code on the ECHONETAPIClient.
You won't be able to use aioudp as its not configured for multicast. You have to use the modified version I added to master.
From this: server = ECHONETAPIClient(server=udp, loop=loop) To this: server = ECHONETAPIClient(udp)
Now it works for the time being. I will start debug after tomorrow.
@scottyphillips I was able to update the HA entities immediately by subscribing to the data update.
I will brush up the code and send you a PR. 😄
I’ll have to release a new version of Pychonet then with the revised multicast code.
Scott
Get Outlook for iOShttps://aka.ms/o0ukef
From: Naoki Sawada @.> Sent: Monday, June 20, 2022 7:05:14 PM To: scottyphillips/pychonet @.> Cc: Scott Phillips @.>; Mention @.> Subject: Re: [scottyphillips/pychonet] Listen to UDP port 3610 (Issue #40)
@scottyphillipshttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fscottyphillips&data=05%7C01%7C%7C87a94650be424cbbc44d08da529bfd27%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637913127170805188%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=PTH8T1DYk18sVon9NGEZED1V8%2FDv7Qf0W1bY%2FY1LN2U%3D&reserved=0 I was able to update the HA entities immediately by subscribing to the data update.
I will brush up the code and send you a PR. 😄
— Reply to this email directly, view it on GitHubhttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fscottyphillips%2Fpychonet%2Fissues%2F40%23issuecomment-1160176738&data=05%7C01%7C%7C87a94650be424cbbc44d08da529bfd27%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637913127170961454%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=KHy%2FgHxVCBX0rYRnkI5%2FLxm%2FnrAVgp8%2Bh9n9oY5muTI%3D&reserved=0, or unsubscribehttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAGHQNDH32GLERMTLVSM7MO3VQAX4VANCNFSM5ZBPRULA&data=05%7C01%7C%7C87a94650be424cbbc44d08da529bfd27%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637913127170961454%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=mTadLUOEGG0GmVEQtv5D5mSd5PM8G67GkKj1B5SltGM%3D&reserved=0. You are receiving this because you were mentioned.Message ID: @.***>
I'll let you know my branch I'm working on.
I updated climate.py but, it has not been verified.
@scottyphillips Is there a way to tell if the data received in echonetMessageReceived()
is due to push or polling?
Before we stop polling, we need to make sure we get the response in push.
It was possible by checking the TID
.
This async_callback functionality is causing all kinds of issues.
For Example in home assistant, when I try to set a fan speed or configure a HVAC mode, it causes chaos. It keeps polluting the update data with "invalid setting" and I am struggling to figure out why.
If we cant resolve this, then I am going to revert all these updates because this push functionality is not really fit for purpose at the moment and is causing more problems then its solving.
See example below when I try to set the fan speed to 'low' inside the climate entity:
2022-07-09 10:38:26 DEBUG (MainThread) [custom_components.echonetlite.climate] Updated fan mode is: low
2022-07-09 10:38:26 DEBUG (MainThread) [custom_components.echonetlite]
ECHONETlite polling update data:
- Operation status (0x80): Off
- Operation status (0x80): Off
- Air flow rate setting (0xa0): Invalid setting
- Operation mode setting (0xb0): invalid_setting
- Set temperature value (0xb3): 19
- Measured value of room temperature (0xbb): 18
- Measured outdoor air temperature (0xbe): 10
2022-07-09 10:38:26 DEBUG (MainThread) [custom_components.echonetlite.climate] Called async_update_callback on boof.
Changed: True
Update data: {128: 'Off', 160: 'Invalid setting', 176: 'invalid_setting', 179: 19, 187: 18, 190: 10}
Old data: {128: 'Off', 160: 'medium-high', 176: 'invalid_setting', 179: 19, 187: 18, 190: 10}
2022-07-09 10:38:26 DEBUG (MainThread) [custom_components.echonetlite.climate] Updated fan mode is:
2022-07-09 10:38:26 ERROR (MainThread) [homeassistant.components.websocket_api.http.connection] [140027908565472] ''
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 193, in handle_call_service
await hass.services.async_call(
File "/usr/src/homeassistant/homeassistant/core.py", line 1713, in async_call
task.result()
File "/usr/src/homeassistant/homeassistant/core.py", line 1750, in _execute_service
await cast(Callable[[ServiceCall], Awaitable[None]], handler.job.target)(
File "/usr/src/homeassistant/homeassistant/helpers/entity_component.py", line 204, in handle_service
await service.entity_service_call(
File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 680, in entity_service_call
future.result() # pop exception if have
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 930, in async_request_call
await coro
File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 717, in _handle_entity_call
await result
File "/config/custom_components/echonetlite/climate.py", line 240, in async_set_fan_mode
await self._connector._instance.setFanSpeed(fan_mode)
File "/usr/local/lib/python3.10/site-packages/pychonet/HomeAirConditioner.py", line 318, in setFanSpeed
return self.setMessage(ENL_FANSPEED, FAN_SPEED[fan_speed])
KeyError: ''
I got it. Let's move to #98@echonetlite_homeassistant.
ECHONET Lite can be notified of device changes by listening to UDP port 3610 at the multicast address 224.0.23.0.
For example, turning
0x90
to on (0x41
) / off (0x42
) on a device with0x02
-0x72
, it will get following data.I think that you can use this to update the device changes immediately. But I'm not sure how to implement as server functionality in Python.
Do we have a chance to implement this?