search

spacemanspiff2007 / SmlLib

A library for the SML (Smart Message Language) protocol
GNU General Public License v3.0
28 stars 9 forks source link

CRC Errors with Holley DTZ541 via RS485 #22

Open webratz opened 3 weeks ago

webratz commented 3 weeks ago

Hey,

after having only used the optical interface so far, I tested today a new Meter (Holley DTZ541) and connected it via a RS485 to USB Serial Adapter.

Every single Frame shows an CRC Error, so I initially thought something is completely off. Out of curiousity I cloned the lib and removed the CRC check to see if there is any useful data at all, and it turns out: The values look actually pretty sane.

{
'1.8.0': 1234, // adjusted to not show real one, but matched actual values on display
'14.7.0': 49.9,
'16.7.0': 259,
'2.8.0': 5678, // adjusted to not show real one , but matched actual values on display
'31.7.0': 0.69,
'32.7.0': 234.6,
'51.7.0': 0.21,
'52.7.0': 233.8,
'71.7.0': 0.58,
'72.7.0': 233.9,
'81.7.1': 120,
'81.7.15': 296,
'81.7.2': 239,
'81.7.26': 317,
'81.7.4': 326 
}

Sample output of a CRC Error

(b'\x1b\x1b\x1b\x1b\x01\x01\x01\x01v\x04\x00\x00\x01b\x00b\x00re\x00\x00\x01\x01v\x01\x01\x07\x00\x00\nWXR\x0b\n\x01HLY\x02\x00\x01Y\xbb\x01\x01cT}\x00v\x04\x00\x00\x02b\x00b\x00re\x00\x00\x07\x01w\x01\x0b\n\x01HLY\x02\x00\x01Y\xbb\x01\x01\xf1\x0ew\x07\x01\x00`2\x01\x01\x01\x01\x01\x01\x04HLY\x01w\x07\x01\x00`\x01\x00\xff\x01\x01\x01\x01\x0b\n\x01HLY\x02\x00\x01Y\xbb\x01w\x07\x01\x00\x01\x08\x00\xffe\x00\x1c\x01\x04e\nWXSb\x1eR\xffe\x06o\x04\xab\x01w\x07\x01\x00\x02\x08\x00\xffe\x00\x1c\x01\x04e\nWXSb\x1eR\xffe\x08\x1c\xb4T\x01w\x07\x01\x00\x10\x07\x00\xff\x01\x01b\x1bR\x00S\x00\xbc\x01w\x07\x01\x00 \x07\x00\xff\x01\x01b#R\xffc\t8\x01w\x07\x01\x004\x07\x00\xff\x01\x01b#R\xffc\t)\x01w\x07\x01\x00H\x07\x00\xff\x01\x01b#R\xffc\t$\x01w\x07\x01\x00\x1f\x07\x00\xff\x01\x01b!R\xfebE\x01w\x07\x01\x003\x07\x00\xff\x01\x01b!R\xfeb\x15\x01w\x07\x01\x00G\x07\x00\xff\x01\x01b!R\xfeb\x1a\x01w\x07\x01\x00Q\x07\x01\xff\x01\x01b\x08R\x00bx\x01w\x07\x01\x00Q\x07\x02\xff\x01\x01b\x08R\x00b\xf0\x01w\x07\x01\x00Q\x07\x04\xff\x01\x01b\x08R\x00c\x01F\x01w\x07\x01\x00Q\x07\x0f\xff\x01\x01b\x08R\x00c\x01)\x01w\x07\x01\x00Q\x07\x1a\xff\x01\x01b\x08R\x00c\x01(\x01w\x07\x01\x00\x0e\x07\x00\xff\x01\x01b,R\xffc\x01\xf4\x01w\x07\x01\x00\x01\x08\x00`\x01\x01b\x1eR\x02b)\x01w\x07\x01\x00\x01\x08\x00a\x01\x01b\x1eR\x02c\x01e\x01w\x07\x01\x00\x01\x08\x00b\x01\x01b\x1eR\x02c\x05\x05\x01w\x07\x01\x00\x01\x08\x00c\x01\x01b\x1eR\x02c`\r\x01w\x07\x01\x00\x01\x08\x00d\x01\x01b\x1eR\x02e\x00\x01\xa5\xa2\x01w\x07\x01\x00\x02\x08\x00`\x01\x01b\x1eR\x02b7\x01w\x07\x01\x00\x02\x08\x00a\x01\x01b\x1eR\x02c\x01X\x01w\x07\x01\x00\x02\x08\x00b\x01\x01b\x1eR\x02c\t\t\x01w\x07\x01\x00\x02\x08\x00c\x01\x01b\x1eR\x02cJ\xec\x01w\x07\x01\x00\x02\x08\x00d\x01\x01b\x1eR\x02e\x00\x02\x13\xa2\x01w\x07\x01\x00\x00\x02\x00\x00\x01\x01\x01\x01\t1.02.007\x01w\x07\x01\x00`Z\x02\x01\x01\x01\x01\x01\x05A01A\x01w\x07\x01\x00`\x05\x00\xff\x01\x01\x01\x01e\x00\x1c\x01\x04\x01\x01\x01c3a\x00v\x04\x00\x00\x03b\x00b\x00re\x00\x00\x02\x01q\x01c\xeb\xf4\x00\x00\x1b\x1b\x1b\x1b\x1a\x01u\x02', 29954, 42739)

I'm opening the serial port in exclusive mode, so I'm assuming that there is no other programs accessing it serial.Serial(self.serial_device, baudrate=9600, bytesize=8, parity='N', stopbits=1, timeout=5, exclusive=True)

Do you have any hints about how I could go on figuring out the reason for the CRC errors? Might the manufacturer even calculate them different, or is maybe the RS485 Adapater changing something.

spacemanspiff2007 commented 3 weeks ago

I think the Holley uses a slightly different CRC-Calculation: https://github.com/volkszaehler/libsml/issues/75#issuecomment-533901295 CRC-16/KERMIT seems to be initialized differently to CRC-16/X-25.

So for now the easiest way would be to just ignore the CRC errors or monkeypatch the crc function call.

webratz commented 3 weeks ago

Would you be open for a MR that allows disabling the CRC check? I might also be able to check if I could get the workaround that volkszaehler implemented ported to smllib

spacemanspiff2007 commented 3 weeks ago

A quick search brought up this. Can you replace the crc function with this and see if it works as expected?

def crc16(data: bytes):
    table = [ 
        0x0000, 0x1021, 0x2042, 0x3063, 0x4084, 0x50A5, 0x60C6, 0x70E7, 0x8108, 0x9129, 0xA14A, 0xB16B, 0xC18C, 0xD1AD, 0xE1CE, 0xF1EF,
        0x1231, 0x0210, 0x3273, 0x2252, 0x52B5, 0x4294, 0x72F7, 0x62D6, 0x9339, 0x8318, 0xB37B, 0xA35A, 0xD3BD, 0xC39C, 0xF3FF, 0xE3DE,
        0x2462, 0x3443, 0x0420, 0x1401, 0x64E6, 0x74C7, 0x44A4, 0x5485, 0xA56A, 0xB54B, 0x8528, 0x9509, 0xE5EE, 0xF5CF, 0xC5AC, 0xD58D,
        0x3653, 0x2672, 0x1611, 0x0630, 0x76D7, 0x66F6, 0x5695, 0x46B4, 0xB75B, 0xA77A, 0x9719, 0x8738, 0xF7DF, 0xE7FE, 0xD79D, 0xC7BC,
        0x48C4, 0x58E5, 0x6886, 0x78A7, 0x0840, 0x1861, 0x2802, 0x3823, 0xC9CC, 0xD9ED, 0xE98E, 0xF9AF, 0x8948, 0x9969, 0xA90A, 0xB92B,
        0x5AF5, 0x4AD4, 0x7AB7, 0x6A96, 0x1A71, 0x0A50, 0x3A33, 0x2A12, 0xDBFD, 0xCBDC, 0xFBBF, 0xEB9E, 0x9B79, 0x8B58, 0xBB3B, 0xAB1A,
        0x6CA6, 0x7C87, 0x4CE4, 0x5CC5, 0x2C22, 0x3C03, 0x0C60, 0x1C41, 0xEDAE, 0xFD8F, 0xCDEC, 0xDDCD, 0xAD2A, 0xBD0B, 0x8D68, 0x9D49,
        0x7E97, 0x6EB6, 0x5ED5, 0x4EF4, 0x3E13, 0x2E32, 0x1E51, 0x0E70, 0xFF9F, 0xEFBE, 0xDFDD, 0xCFFC, 0xBF1B, 0xAF3A, 0x9F59, 0x8F78,
        0x9188, 0x81A9, 0xB1CA, 0xA1EB, 0xD10C, 0xC12D, 0xF14E, 0xE16F, 0x1080, 0x00A1, 0x30C2, 0x20E3, 0x5004, 0x4025, 0x7046, 0x6067,
        0x83B9, 0x9398, 0xA3FB, 0xB3DA, 0xC33D, 0xD31C, 0xE37F, 0xF35E, 0x02B1, 0x1290, 0x22F3, 0x32D2, 0x4235, 0x5214, 0x6277, 0x7256,
        0xB5EA, 0xA5CB, 0x95A8, 0x8589, 0xF56E, 0xE54F, 0xD52C, 0xC50D, 0x34E2, 0x24C3, 0x14A0, 0x0481, 0x7466, 0x6447, 0x5424, 0x4405,
        0xA7DB, 0xB7FA, 0x8799, 0x97B8, 0xE75F, 0xF77E, 0xC71D, 0xD73C, 0x26D3, 0x36F2, 0x0691, 0x16B0, 0x6657, 0x7676, 0x4615, 0x5634,
        0xD94C, 0xC96D, 0xF90E, 0xE92F, 0x99C8, 0x89E9, 0xB98A, 0xA9AB, 0x5844, 0x4865, 0x7806, 0x6827, 0x18C0, 0x08E1, 0x3882, 0x28A3,
        0xCB7D, 0xDB5C, 0xEB3F, 0xFB1E, 0x8BF9, 0x9BD8, 0xABBB, 0xBB9A, 0x4A75, 0x5A54, 0x6A37, 0x7A16, 0x0AF1, 0x1AD0, 0x2AB3, 0x3A92,
        0xFD2E, 0xED0F, 0xDD6C, 0xCD4D, 0xBDAA, 0xAD8B, 0x9DE8, 0x8DC9, 0x7C26, 0x6C07, 0x5C64, 0x4C45, 0x3CA2, 0x2C83, 0x1CE0, 0x0CC1,
        0xEF1F, 0xFF3E, 0xCF5D, 0xDF7C, 0xAF9B, 0xBFBA, 0x8FD9, 0x9FF8, 0x6E17, 0x7E36, 0x4E55, 0x5E74, 0x2E93, 0x3EB2, 0x0ED1, 0x1EF0
    ]

    crc = 0xFFFF
    for byte in data:
        crc = (crc << 8) ^ table[(crc >> 8) ^ byte]
        crc &= 0xFFFF
    return crc

I am not sure if disabling the crc check all together is a good idea. My meter sporadically issues corrupt frames which will then result in another runtime exception which can not be caught as easily as the CrcError. However I think it makes sense to make that smllib supports additional algorithms so I'll think of something. I'd like to do some additional cleanup, so I'd prefer to do it myself but it might take some time. Is the manual edit something you could live with?

webratz commented 3 weeks ago

This does not seem to work right away. I'll probably need some time to check in detail

File "/opt/sml2mqtt/venv/lib/python3.11/site-packages/smllib/reader.py", line 56, in get_frame
    crc_calc = crc16(msg[:-2])
               ^^^^^^^^^^^^^^^
  File "/opt/sml2mqtt/venv/lib/python3.11/site-packages/smllib/crc.py", line 69, in crc16
    crc = (crc << 8) ^ table[(crc >> 8) ^ byte]
                       ~~~~~^^^^^^^^^^^^^^^^^^^
IndexError: list index out of range

Also this error seems to hit me https://github.com/volkszaehler/libsml/pull/76/files#diff-c1f2ae13f8f0c53b6842f00718fd75d67895e04f1bc7aa66857096f1e6cab1bbR41 so I guess I'll need to add this also, even when skipping the CRC checks

webratz commented 3 weeks ago

It seems like there is error at times, when the CRC check is not there, catching issues

[2024-08-21 19:27:18,003] [sml.ttyUSB0       ] INFO     | get_obis failed - try parsing frame
[2024-08-21 19:27:18,008] [sml.ttyUSB0       ] INFO     | Received Frame
[2024-08-21 19:27:18,009] [sml.ttyUSB0       ] INFO     |  -> b'7604000001620062007265000001017601010700000a5808070b0a01484c5902000159bb01016397470076040000026200620072650000070177010b0a01484c5902000159bb0101f10e77070100603201010101010104484c590177070100600100ff010101010b0a01484c5902000159bb0177070100010800ff65001c1904650a580807621e52ff65066f5c380177070100020800ff65001c1904650a580807621e52ff65081cc5f30177070100100700ff0101621b520053fdda0177070100200700ff0101622352ff6309450177070100340700ff0101622352ff63093f0177070100480700ff0101622352ff63094201770701001f0700ff0101622152fe62f90177070100330700ff0101622152fe62120177070100470700ff0101622152fe621b0177070100510701ff01016208520062770177070100510702ff01016208520062ef0177070100510704ff01016208520062b2017707010051070fff01016208520063011f017707010051071aff01016208520063012801770701000e0700ff0101622c52ff6301f40177070100010800600101621e520262290177070100010800610101621e52026301650177070100010800620101621e52026305050177070100010800630101621e520263600d0177070100010800640101621e5202650001a5b90177070100020800600101621e520262370177070100020800610101621e52026301580177070100020800620101621e5202630909017707010002080063010162fe1b1b1b1b010101017604000001620062007265000001017601010700000a58080a0b0a01484c5902000159bb010163f5770076040000026200620072650000070177010b0a01484c5902000159bb0101f10e77070100603201010101010104484c590177070100600100ff010101010b0a01484c5902000159bb0177070100010800ff65001c1904650a58080a621e52ff65066f5c380177070100020800ff65001c1904650a58080a621e52ff65081cc5f80177070100100700ff0101621b520053fdd80177070100200700ff0101622352ff6309440177070100340700ff0101622352ff63093f0177070100480700ff0101622352ff63094201770701001f0700ff0101622152fe62fb0177070100330700ff0101622152fe62120177070100470700ff0101622152fe621b0177070100510701ff01016208520062770177070100510702ff01016208520062f00177070100510704ff01016208520062b8017707010051070fff010162085200630120017707010051071aff01016208520063012801770701000e0700ff0101622c52ff6301f40177070100010800600101621e520262290177070100010800610101621e52026301650177070100010800620101621e52026305050177070100010800630101621e520263600d0177070100010800640101621e5202650001a5b90177070100020800600101621e520262370177070100020800610101621e52026301580177070100020800620101621e52026309090177070100020800630101621e5202634aec0177070100020800640101621e520265000213a70177070100000200000101010109312e30322e3030370177070100605a02010101010105413031410177070100600500ff0101010165001c1904010101632b0700760400000362006200726500000201710163ebf400'
[2024-08-21 19:27:18,010] [sml.ttyUSB0       ] DEBUG    | Exception <class 'ValueError'>: "Unknown data type: 10!"
[2024-08-21 19:27:18,018] [sml.ttyUSB0       ] ERROR    | Traceback (most recent call last):
[2024-08-21 19:27:18,019] [sml.ttyUSB0       ] ERROR    |   File "/opt/sml2mqtt/venv/lib/python3.11/site-packages/sml2mqtt/const/sml_helpers.py", line 40, in get_frame_values
[2024-08-21 19:27:18,020] [sml.ttyUSB0       ] ERROR    |     sml_objs = self.get_obis()  # type: list[SmlListEntry]
[2024-08-21 19:27:18,020] [sml.ttyUSB0       ] ERROR    |                ^^^^^^^^^^^^^^^
[2024-08-21 19:27:18,021] [sml.ttyUSB0       ] ERROR    |   File "/opt/sml2mqtt/venv/lib/python3.11/site-packages/smllib/sml_frame.py", line 127, in get_obis
[2024-08-21 19:27:18,021] [sml.ttyUSB0       ] ERROR    |     self._parse_msg(data)
[2024-08-21 19:27:18,021] [sml.ttyUSB0       ] ERROR    |   File "/opt/sml2mqtt/venv/lib/python3.11/site-packages/smllib/sml_frame.py", line 112, in _parse_msg
[2024-08-21 19:27:18,022] [sml.ttyUSB0       ] ERROR    |     _lst[i] = v = self.get_value()
[2024-08-21 19:27:18,022] [sml.ttyUSB0       ] ERROR    |                   ^^^^^^^^^^^^^^^^
[2024-08-21 19:27:18,022] [sml.ttyUSB0       ] ERROR    |   File "/opt/sml2mqtt/venv/lib/python3.11/site-packages/smllib/sml_frame.py", line 89, in get_value
[2024-08-21 19:27:18,023] [sml.ttyUSB0       ] ERROR    |     raise ValueError(msg)
[2024-08-21 19:27:18,023] [sml.ttyUSB0       ] ERROR    | ValueError: Unknown data type: 10!
[2024-08-21 19:27:18,024] [sml.ttyUSB0       ] ERROR    | 
[2024-08-21 19:27:18,024] [sml.ttyUSB0       ] ERROR    | During handling of the above exception, another exception occurred:
[2024-08-21 19:27:18,024] [sml.ttyUSB0       ] ERROR    | 
[2024-08-21 19:27:18,025] [sml.ttyUSB0       ] ERROR    | Traceback (most recent call last):
[2024-08-21 19:27:18,025] [sml.ttyUSB0       ] ERROR    |   File "/opt/sml2mqtt/venv/lib/python3.11/site-packages/sml2mqtt/sml_device/sml_device.py", line 115, in on_source_data
[2024-08-21 19:27:18,025] [sml.ttyUSB0       ] ERROR    |     self.frame_handler(frame)
[2024-08-21 19:27:18,025] [sml.ttyUSB0       ] ERROR    |   File "/opt/sml2mqtt/venv/lib/python3.11/site-packages/sml2mqtt/sml_device/sml_device.py", line 150, in process_frame
[2024-08-21 19:27:18,026] [sml.ttyUSB0       ] ERROR    |     frame_values = frame.get_frame_values(self.log)
[2024-08-21 19:27:18,026] [sml.ttyUSB0       ] ERROR    |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-08-21 19:27:18,026] [sml.ttyUSB0       ] ERROR    |   File "/opt/sml2mqtt/venv/lib/python3.11/site-packages/sml2mqtt/const/sml_helpers.py", line 45, in get_frame_values
[2024-08-21 19:27:18,026] [sml.ttyUSB0       ] ERROR    |     for msg in self.parse_frame():
[2024-08-21 19:27:18,027] [sml.ttyUSB0       ] ERROR    |                ^^^^^^^^^^^^^^^^^^
[2024-08-21 19:27:18,027] [sml.ttyUSB0       ] ERROR    |   File "/opt/sml2mqtt/venv/lib/python3.11/site-packages/smllib/sml_frame.py", line 104, in parse_frame
[2024-08-21 19:27:18,027] [sml.ttyUSB0       ] ERROR    |     val = self._parse_msg(self.get_value())
[2024-08-21 19:27:18,028] [sml.ttyUSB0       ] ERROR    |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-08-21 19:27:18,028] [sml.ttyUSB0       ] ERROR    |   File "/opt/sml2mqtt/venv/lib/python3.11/site-packages/smllib/sml_frame.py", line 114, in _parse_msg
[2024-08-21 19:27:18,028] [sml.ttyUSB0       ] ERROR    |     self._parse_msg(v)
[2024-08-21 19:27:18,028] [sml.ttyUSB0       ] ERROR    |   File "/opt/sml2mqtt/venv/lib/python3.11/site-packages/smllib/sml_frame.py", line 114, in _parse_msg
[2024-08-21 19:27:18,029] [sml.ttyUSB0       ] ERROR    |     self._parse_msg(v)
[2024-08-21 19:27:18,029] [sml.ttyUSB0       ] ERROR    |   File "/opt/sml2mqtt/venv/lib/python3.11/site-packages/smllib/sml_frame.py", line 114, in _parse_msg
[2024-08-21 19:27:18,029] [sml.ttyUSB0       ] ERROR    |     self._parse_msg(v)
[2024-08-21 19:27:18,030] [sml.ttyUSB0       ] ERROR    |   [Previous line repeated 1 more time]
[2024-08-21 19:27:18,030] [sml.ttyUSB0       ] ERROR    |   File "/opt/sml2mqtt/venv/lib/python3.11/site-packages/smllib/sml_frame.py", line 112, in _parse_msg
[2024-08-21 19:27:18,030] [sml.ttyUSB0       ] ERROR    |     _lst[i] = v = self.get_value()
[2024-08-21 19:27:18,030] [sml.ttyUSB0       ] ERROR    |                   ^^^^^^^^^^^^^^^^
[2024-08-21 19:27:18,031] [sml.ttyUSB0       ] ERROR    |   File "/opt/sml2mqtt/venv/lib/python3.11/site-packages/smllib/sml_frame.py", line 89, in get_value
[2024-08-21 19:27:18,031] [sml.ttyUSB0       ] ERROR    |     raise ValueError(msg)
[2024-08-21 19:27:18,031] [sml.ttyUSB0       ] ERROR    | ValueError: Unknown data type: 10!
[2024-08-21 19:27:18,032] [sml.ttyUSB0.status] INFO     | ERROR
[2024-08-21 19:27:18,034] [sml.shutdown      ] INFO     | Shutting down ...
spacemanspiff2007 commented 3 weeks ago

The bitwise and of the crc belongs in the for loop:

    for byte in data:
        crc = (crc << 8) ^ table[(crc >> 8) ^ byte]
        crc &= 0xFFFF
    return crc

Does crc work when you do it like this? I think it makes sense to fix crc first because the meter will issue corrupt frames and then your program will crash. Only when this is fixed it makes sense to try the workarounds. It would be nice if you could provide the received frames which require the workaround so I can add them to the test cases.

webratz commented 3 weeks ago

Sadly that still does not work.

Not sure if its the right format, but an example frame looks like this

b'v\x04\x00\x00\x01b\x00b\x00re\x00\x00\x01\x01v\x01\x01\x07\x00\x00\nX;\x80\x0b\n\x01HLY\x02\x00\x01Y\xbb\x01\x01cV]\x00v\x04\x00\x00\x02b\x00b\x00re\x00\x00\x07\x01w\x01\x0b\n\x01HLY\x02\x00\x01Y\xbb\x01\x01\xf1\x0ew\x07\x01\x00`2\x01\x01\x01\x01\x01\x01\x04HLY\x01w\x07\x01\x00`\x01\x00\xff\x01\x01\x01\x01\x0b\n\x01HLY\x02\x00\x01Y\xbb\x01w\x07\x01\x00\x01\x08\x00\xffe\x00\x1c\x19\x04e\nX;\x80b\x1eR\xffe\x06o\\8\x01w\x07\x01\x00\x02\x08\x00\xffe\x00\x1c\x19\x04e\nX;\x80b\x1eR\xffe\x08\x1db\xff\x01w\x07\x01\x00\x10\x07\x00\xff\x01\x01b\x1bR\x00S\xf9\xbe\x01w\x07\x01\x00 \x07\x00\xff\x01\x01b#R\xffc\tv\x01w\x07\x01\x004\x07\x00\xff\x01\x01b#R\xffc\tO\x01w\x07\x01\x00H\x07\x00\xff\x01\x01b#R\xffc\tu\x01w\x07\x01\x00\x1f\x07\x00\xff\x01\x01b!R\xfec\x02\xd5\x01w\x07\x01\x003\x07\x00\xff\x01\x01b!R\xfeb"\x01w\x07\x01\x00G\x07\x00\xff\x01\x01b!R\xfeb2\x01w\x07\x01\x00Q\x07\x01\xff\x01\x01b\x08R\x00bx\x01w\x07\x01\x00Q\x07\x02\xff\x01\x01b\x08R\x00b\xf0\x01w\x07\x01\x00Q\x07\x04\xff\x01\x01b\x08R\x00b\xb4\x01w\x07\x01\x00Q\x07\x0f\xff\x01\x01b\x08R\x00b\x0c\x01w\x07\x01\x00Q\x07\x1a\xff\x01\x01b\x08R\x00c\x015\x01w\x07\x01\x00\x0e\x07\x00\xff\x01\x01b,R\xffc\x01\xf4\x01w\x07\x01\x00\x01\x08\x00`\x01\x01b\x1eR\x02b)\x01w\x07\x01\x00\x01\x08\x00a\x01\x01b\x1eR\x02c\x01e\x01w\x07\x01\x00\x01\x08\x00b\x01\x01b\x1eR\x02c\x05\x05\x01w\x07\x01\x00\x01\x08\x00c\x01\x01b\x1eR\x02c`\r\x01w\x07\x01\x00\x01\x08\x00d\x01\x01b\x1eR\x02e\x00\x01\xa5\xb9\x01w\x07\x01\x00\x02\x08\x00`\x01\x01b\x1eR\x02b7\x01w\x07\x01\x00\x02\x08\x00a\x01\x01b\x1eR\x02c\x01X\x01w\x07\x01\x00\x02\x08\x00b\x01\x01b\x1eR\x02c\t\t\x01w\x07\x01\x00\x02\x08\x00c\x01\x01b\x1eR\x02cJ\xec\x01w\x07\x01\x00\x02\x08\x00d\x01\x01b\x1eR\x02e\x00\x02\x13\xcf\x01w\x07\x01\x00\x00\x02\x00\x00\x01\x01\x01\x01\t1.02.007\x01w\x07\x01\x00`Z\x02\x01\x01\x01\x01\x01\x05A01A\x01w\x07\x01\x00`\x05\x00\xff\x01\x01\x01\x01e\x00\x1c\x19\x04\x01\x01\x01c\xb0l\x00v\x04\x00\x00\x03b\x00b\x00re\x00\x00\x02\x01q\x01c\xeb\xf4\x00'
webratz commented 3 weeks ago

CRC16 KERMIT using https://github.com/Nicoretti/crc/blob/master/src/crc/_crc.py#L529 for test purposes works fine

config = Configuration(
    width=16,
    polynomial=0x1021,
    init_value=0x0000,
    final_xor_value=0x0000 ,
    reverse_input=True,
    reverse_output=True,
)
webratz commented 3 weeks ago 

def crc16_kermit(data: bytes):
    table = [
        0x0000, 0x1189, 0x2312, 0x329B, 0x4624, 0x57AD, 0x6536, 0x74BF,
        0x8C48, 0x9DC1, 0xAF5A, 0xBED3, 0xCA6C, 0xDBE5, 0xE97E, 0xF8F7,
        0x1081, 0x0108, 0x3393, 0x221A, 0x56A5, 0x472C, 0x75B7, 0x643E,
        0x9CC9, 0x8D40, 0xBFDB, 0xAE52, 0xDAED, 0xCB64, 0xF9FF, 0xE876,
        0x2102, 0x308B, 0x0210, 0x1399, 0x6726, 0x76AF, 0x4434, 0x55BD,
        0xAD4A, 0xBCC3, 0x8E58, 0x9FD1, 0xEB6E, 0xFAE7, 0xC87C, 0xD9F5,
        0x3183, 0x200A, 0x1291, 0x0318, 0x77A7, 0x662E, 0x54B5, 0x453C,
        0xBDCB, 0xAC42, 0x9ED9, 0x8F50, 0xFBEF, 0xEA66, 0xD8FD, 0xC974,
        0x4204, 0x538D, 0x6116, 0x709F, 0x0420, 0x15A9, 0x2732, 0x36BB,
        0xCE4C, 0xDFC5, 0xED5E, 0xFCD7, 0x8868, 0x99E1, 0xAB7A, 0xBAF3,
        0x5285, 0x430C, 0x7197, 0x601E, 0x14A1, 0x0528, 0x37B3, 0x263A,
        0xDECD, 0xCF44, 0xFDDF, 0xEC56, 0x98E9, 0x8960, 0xBBFB, 0xAA72,
        0x6306, 0x728F, 0x4014, 0x519D, 0x2522, 0x34AB, 0x0630, 0x17B9,
        0xEF4E, 0xFEC7, 0xCC5C, 0xDDD5, 0xA96A, 0xB8E3, 0x8A78, 0x9BF1,
        0x7387, 0x620E, 0x5095, 0x411C, 0x35A3, 0x242A, 0x16B1, 0x0738,
        0xFFCF, 0xEE46, 0xDCDD, 0xCD54, 0xB9EB, 0xA862, 0x9AF9, 0x8B70,
        0x8408, 0x9581, 0xA71A, 0xB693, 0xC22C, 0xD3A5, 0xE13E, 0xF0B7,
        0x0840, 0x19C9, 0x2B52, 0x3ADB, 0x4E64, 0x5FED, 0x6D76, 0x7CFF,
        0x9489, 0x8500, 0xB79B, 0xA612, 0xD2AD, 0xC324, 0xF1BF, 0xE036,
        0x18C1, 0x0948, 0x3BD3, 0x2A5A, 0x5EE5, 0x4F6C, 0x7DF7, 0x6C7E,
        0xA50A, 0xB483, 0x8618, 0x9791, 0xE32E, 0xF2A7, 0xC03C, 0xD1B5,
        0x2942, 0x38CB, 0x0A50, 0x1BD9, 0x6F66, 0x7EEF, 0x4C74, 0x5DFD,
        0xB58B, 0xA402, 0x9699, 0x8710, 0xF3AF, 0xE226, 0xD0BD, 0xC134,
        0x39C3, 0x284A, 0x1AD1, 0x0B58, 0x7FE7, 0x6E6E, 0x5CF5, 0x4D7C,
        0xC60C, 0xD785, 0xE51E, 0xF497, 0x8028, 0x91A1, 0xA33A, 0xB2B3,
        0x4A44, 0x5BCD, 0x6956, 0x78DF, 0x0C60, 0x1DE9, 0x2F72, 0x3EFB,
        0xD68D, 0xC704, 0xF59F, 0xE416, 0x90A9, 0x8120, 0xB3BB, 0xA232,
        0x5AC5, 0x4B4C, 0x79D7, 0x685E, 0x1CE1, 0x0D68, 0x3FF3, 0x2E7A,
        0xE70E, 0xF687, 0xC41C, 0xD595, 0xA12A, 0xB0A3, 0x8238, 0x93B1,
        0x6B46, 0x7ACF, 0x4854, 0x59DD, 0x2D62, 0x3CEB, 0x0E70, 0x1FF9,
        0xF78F, 0xE606, 0xD49D, 0xC514, 0xB1AB, 0xA022, 0x92B9, 0x8330,
        0x7BC7, 0x6A4E, 0x58D5, 0x495C, 0x3DE3, 0x2C6A, 0x1EF1, 0x0F78
    ]

    crc = 0x0000  # Initial value for CRC16 KERMIT
    for byte in data:
        crc = (crc >> 8) ^ table[(crc ^ byte) & 0xFF]
    return crc

this seems to work also, not sure if I trust it, asked the AI to generate it

spacemanspiff2007 commented 3 weeks ago

Do you still get errors with the CRC check implemented? Or is the complete frame invalid?

webratz commented 2 weeks ago

With the new CRC check implemented things work fine now. There is the occasional corrupt frame, but in general stable

spacemanspiff2007 commented 2 weeks ago

So do I understand it correctly that no further workaround is needed for you? It's just the modified CRC check?

webratz commented 2 weeks ago

Exactly, all I implemented was the CRC16 Kermit, and so far only manually patched in. So adding that to the lib and either allow it to be configured, or fall back to it like libsml does, would allow everyone with the Holley DTZ541 to use it.