Reduced support for non-evohome controllers

Schorsch99 commented 3 years ago

Hi zxdavb,

Maybe you can help me with this. I am getting some good decoded data, but also very many error messages (mainly "coding error" and "validation error"). Could this be something in my configuration/hardware, or is it code-related and the decoding of these specific messages is not fully implemented yet?

Here are a few examples:

2021-05-27T08:35:15.785452 090  I 108 --:------ --:------ 12:126457 2309 006 0107D0020708 < Validation error: expecting length 3
Traceback (most recent call last):
  File "/home/pi/ramses_rf/ramses_rf/message.py", line 402, in is_valid
    self._payload = payload_parser(self.raw_payload, self)
  File "/home/pi/ramses_rf/ramses_rf/parsers.py", line 265, in wrapper
    result = func(*args, **kwargs)
  File "/home/pi/ramses_rf/ramses_rf/parsers.py", line 1441, in parser_2309
    assert msg.len == 3, "expecting length 3"
AssertionError: expecting length 3

2021-05-27T08:35:15.816891 087  I 110 --:------ --:------ 12:126457 000A 012 010001F40BB8020001F40BB8 < Validation error: expecting length 6
Traceback (most recent call last):
  File "/home/pi/ramses_rf/ramses_rf/message.py", line 402, in is_valid
    self._payload = payload_parser(self.raw_payload, self)
  File "/home/pi/ramses_rf/ramses_rf/parsers.py", line 265, in wrapper
    result = func(*args, **kwargs)
  File "/home/pi/ramses_rf/ramses_rf/parsers.py", line 611, in parser_000a
    assert msg.len == 6, "expecting length 6"
AssertionError: expecting length 6

2021-05-27T08:35:25.702793 064  I --- 00:034798 --:------ 12:126457 1060 003 02FF01 < Coding error
Traceback (most recent call last):
  File "/home/pi/ramses_rf/ramses_rf/message.py", line 402, in is_valid
    self._payload = payload_parser(self.raw_payload, self)
  File "/home/pi/ramses_rf/ramses_rf/parsers.py", line 268, in wrapper
    return {**_idx(payload[:2], msg), **result}
  File "/home/pi/ramses_rf/ramses_rf/parsers.py", line 144, in _idx
    raise TypeError("result={}")
TypeError: result={}

2021-05-27T08:35:34.202872 069  I --- 04:029362 --:------ 12:126457 3150 002 0162 < Coding error
Traceback (most recent call last):
  File "/home/pi/ramses_rf/ramses_rf/message.py", line 402, in is_valid
    self._payload = payload_parser(self.raw_payload, self)
  File "/home/pi/ramses_rf/ramses_rf/parsers.py", line 265, in wrapper
    result = func(*args, **kwargs)
  File "/home/pi/ramses_rf/ramses_rf/parsers.py", line 1640, in parser_3150
    return _parser(payload)  # TODO: check UFC/FC is == CTL/FC
  File "/home/pi/ramses_rf/ramses_rf/parsers.py", line 1634, in _parser
    return {**_idx(seqx[:2], msg), "heat_demand": _percent(seqx[2:])}
  File "/home/pi/ramses_rf/ramses_rf/parsers.py", line 144, in _idx
    raise TypeError("result={}")
TypeError: result={}

Note this device sends a 1F09 packet:

08:35:15.771 ||            | THm:126457 |  I | system_sync      | 000BC2   || {'remaining_seconds': 301.0, '_next_sync': '08:40:16'}

Thanks for checking! :-)

zxdavb commented 3 years ago

Hello, I am sorry - most of my testing is on evohome. I should be able to sort this out... Could we start by you describing your setup? What controller/relay/zones?

Schorsch99 commented 3 years ago

Hello, no worries :-) My "setup" is quite simple:

a cm67z as a 2-zone "controller"
four HR80 (I have a few more, but not installed yet) I do have an additional Evohome Touch controller (the old version with monochrome display), but so far I only powered that up a few times for testing purposes.

zxdavb commented 3 years ago

The cm67m explains everything. It looks like a DTS92, but supports multiple zones.

It would be helpful for you to send me a 24h packet log, and I can dev/test against?

What device are you using to pick up packets?

Schorsch99 commented 3 years ago

Took me a bit to figure out how to create a packet log :-)

Do you need a full 24h log? I'm getting several errors per minute, so an hour should contain about 120 - 240 of these packets.

I'm using evofw3 0.6.14 (Nano & CC1101) hooked up to a RPi4.

zxdavb commented 3 years ago

Evofw3 v7 is out and is strongly recommended.

24h log is better, but sent me whatever you can...

I've done no significant testing against a c67m, and
I won't be able to look at it for 12h anyway, and
previous experience has shown longer logs are best (but >2h would be OK)
some packets are sent less often, hourly, or once/24h

Even better, if you change modes/ set points, etc. During that time.

What is your relay?

Schorsch99 commented 3 years ago

Thanks for the hint, I have now bumped it up to version 0.7.0 :-)

Log is restarted, I'll send you the results tomorrow. I will change some setpoints here and there :-)

I don't have a relay, only the CM67z and the HR80s.

zxdavb commented 3 years ago

Try:

python -O client.py -rr listen /dev/ttyUSB0 -o packet.log

zxdavb commented 3 years ago

So, the issue here is that the cm97z (which 'looks' like a DTS92, a thermostat) is acting a multi-zone controller (just like an evohome controller).

The evohome_rf library was always meant to deal with different types of controllers, but it has been almost exclusively written on & tested against evohome controllers.

Unfortunately, the cm97z may initially look like a thermostat, and only later on act like a controller...

I have tried to work out a way of automatically 'promoting' a device from being a thermostat to a controller, but I believe this will result is too many deep changes...

I had another solution already planned, but this is not imminent.

I will get rid of the parsing errors for now, and then see where we go next.

Schorsch99 commented 3 years ago

Thanks! Now I get no more error messages. But the Terminal output (with the command above for generating packet logs) is truncated:

08:46:12.929 || TRv:000517 |            |  I | temperature      | 000809   || {'temperature': 20.57}
08:46:13.928 || TRv:000517 | THm:126457 |  I | heat_demand      | 017C     || {'parent_idx': '01', 'heat_demand': 
08:46:28.936 || TRV:029390 |            |  I | temperature      | 0007C8   || {'temperature': 19.92}
08:46:29.930 || TRv:000517 | THm:126457 |  I | device_battery   | 01FF01   || {'parent_idx': '01', 'battery_low': 
08:46:29.941 || TRv:000517 |            |  I | device_battery   | 00FF01   || {'battery_low': False, 'battery_leve
08:46:39.435 || TRV:029390 | THm:126457 |  I | setpoint         | 0107D0   || {'parent_idx': '01', 'setpoint': 20.
08:47:06.434 || TRv:034798 |            |  I | temperature      | 000836   || {'temperature': 21.02}
08:47:18.445 || TRV:029362 |            |  I | device_battery   | 00FF01   || {'battery_low': False, 'battery_leve
08:47:41.935 || TRv:034798 |            |  I | temperature      | 000836   || {'temperature': 21.02}
08:48:08.933 || TRV:029362 |            |  I | temperature      | 0007D5   || {'temperature': 20.05}
08:48:11.434 || TRV:029362 | THm:126457 |  I | setpoint         | 0107D0   || {'parent_idx': '01', 'setpoint': 20.
08:48:16.934 || TRV:029362 |            |  I | temperature      | 0007D5   || {'temperature': 20.05}
08:48:23.432 || TRV:029362 | THm:126457 |  I | heat_demand      | 0162     || {'parent_idx': '01', 'heat_demand': 
08:48:51.933 || TRV:029362 | THm:126457 |  I | setpoint         | 0107D0   || {'parent_idx': '01', 'setpoint': 20.

Also gw_evohome_rf crashes immediately:

2021-06-02 08:53:38 |
2021-06-02 08:53:38 | ------------------------------------------------------------------------------------------
2021-06-02 08:53:38 | Devices loaded from 'devices.json' file:
2021-06-02 08:53:38 |    TRv 00:000517 - TRv:000517
2021-06-02 08:53:38 |    TRv 00:034798 - TRv:034798
2021-06-02 08:53:38 |    TRV 04:029362 - TRV:029362
2021-06-02 08:53:38 |    TRV 04:029390 - TRV:029390
2021-06-02 08:53:38 |    HGI 18:000730 - EvoGateway
2021-06-02 08:53:38 | ------------------------------------------------------------------------------------------
2021-06-02 08:53:38 |
2021-06-02 08:53:38 | # evogateway 3.0.0_alpha9
An empty block_list was enabled, so will be ignored
Traceback (most recent call last):
  File "evogateway.py", line 973, in <module>
    asyncio.run(main())
  File "/usr/lib/python3.7/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
    return future.result()
  File "evogateway.py", line 933, in main
    GWY = Gateway(serial_port, **lib_kwargs)    
  File "/home/pi/.local/lib/python3.7/site-packages/ramses_rf/__init__.py", line 103, in __init__
    self.known_devices = load_system_schema(self, **schema)
  File "/home/pi/.local/lib/python3.7/site-packages/ramses_rf/schema.py", line 364, in load_system_schema
    _load_system_schema(gwy, kwargs[SCHEMA])
  File "/home/pi/.local/lib/python3.7/site-packages/ramses_rf/schema.py", line 382, in _load_system_schema
    schema = SYSTEM_SCHEMA(schema)
  File "/home/pi/.local/lib/python3.7/site-packages/voluptuous/schema_builder.py", line 272, in __call__
    return self._compiled([], data)
  File "/home/pi/.local/lib/python3.7/site-packages/voluptuous/validators.py", line 215, in _run
    return self._exec(self._compiled, value, path)
  File "/home/pi/.local/lib/python3.7/site-packages/voluptuous/validators.py", line 267, in _exec
    self.msg, path=path)
  File "/home/pi/.local/lib/python3.7/site-packages/voluptuous/validators.py", line 260, in _exec
    return func(path, v)
  File "/home/pi/.local/lib/python3.7/site-packages/voluptuous/schema_builder.py", line 594, in validate_dict
    return base_validate(path, iteritems(data), out)
  File "/home/pi/.local/lib/python3.7/site-packages/voluptuous/schema_builder.py", line 432, in validate_mapping
    raise er.MultipleInvalid(errors)
voluptuous.error.MultipleInvalid: extra keys not allowed @ data['main_controller']

I don't know if these issues are related in any way.

zxdavb commented 3 years ago

But the Terminal output (with the command above for generating packet logs) is truncated:

This is normal behaviour - if you want to see it all pass the output though a pipe, or send it to a file.

Also gw_evohome_rf crashes immediately:

Seems like your schema is no correctly formatted?

zxdavb commented 3 years ago

Latest status (i.e. ramses_rf 0.10.0) - there remains little support for non-evohome controllers, despite improvements to the parser. This will be addressed in time.

Schorsch99 commented 3 years ago

I understand that, but maybe you could help me with this: with the latest version I get the following error, even before the first signal (from the non-evohome controller) has been received:

pi@iobroker:~/ramses_rf $ python3 -O client.py -rr listen /dev/ttyUSB0 -o packet3.log

client.py: Starting ramses_rf...
11:15:13.172 No allow_list was configured, but one is strongly recommended
11:15:13.180 Not using a device filter (an allow_list is strongly recommended)
2021-06-24T11:15:13.180850 # ramses_rf 0.10.2
2021-06-24T11:15:14.645204  # evofw3 0.7.0
11:15:14.646 Task exception was never retrieved
future: <Task finished coro=<PacketProtocolBase._send_data() done, defined at /home/pi/ramses_rf/ramses_rf/transport.py:412> exception=AttributeError("'NoneType' object has no attribute 'encode'")>
Traceback (most recent call last):
  File "/home/pi/ramses_rf/ramses_rf/transport.py", line 431, in _send_data
    data = bytes(data.encode("ascii"))
AttributeError: 'NoneType' object has no attribute 'encode'
11:15:25.354 || TRv:034798 |            |  I | temperature      | 000897   || {'temperature': 21.99}

From here on output is normal.

zxdavb commented 3 years ago

AttributeError: 'NoneType' object has no attribute 'encode'

This has been fixed - will be pushing an update soon.

Schorsch99 commented 3 years ago

Update works. Thanks! :-)

zxdavb / ramses_rf

Reduced support for non-evohome controllers #30