Error: Error while parsing SML message: Error: Unknown TL-Field 0x36 for OctetString [Offset: 223]!

[nknapp]

Describe the bug

I have an Iskra MT 175 counter and I try to read out the data using an optical IR reader, via USB I haven't entered a PIN in my counter yet, so I don't have access to all values.

When I try to read the message with smartmeter-obis I always get the following message:

Error: Error while parsing SML message: Error: Unknown TL-Field 0x36 for OctetString [Offset: 223]!: Message: 1b1b1b1b0101010176050541733562006200726301017601010501407b3d0b090149534b000403737a0101633f7600760505417336620062007263070177010b090149534b000403737a070100620a7f7f726201650251045876770701014702037f010101010449534b01770701000000097f010101010b090149534b000403737a01770701000108007f650000010201621e520369000000000000122001770701000108017f0101621e520369000000000000122001770701000108027f0101621e520369000000000000000001770701014702057f010101010302763622195d6e03607b6f0647065517504a395d0f1a7b6f7237190112595f437272504c201f5e67705a3530201540202a010101631e730076050541733762006200726302017101637b2e001b1b1b1b1a007e01

The error message is exactly the same, every time I start my program.

To Reproduce

I use the following code to run smartmeter-obis

var SmartmeterObis = require('smartmeter-obis');

var options = {
    'protocol': "SmlProtocol",
    'transport': "SerialRequestResponseTransport",
    'transportSerialPort': "/dev/ttyUSB0",
    'transportSerialBaudrate': 9600,
    'requestInterval': 60,
    'obisNameLanguage': 'de',
    'obisFallbackMedium': 6
};

function displayData(err, obisResult) {
    console.log(err, obisResult)
}

var smTransport = SmartmeterObis.init(options, displayData);

smTransport.process();

setTimeout(smTransport.stop, 60000);

Expected behavior

I would expect to see some kind of message from my smartmeter, but I just see the error

Versions

Using Node.js 16

[nknapp]

I have added some console.log statements into the code and tracked the error down to these SML_ListEntry items:

last valid entry:
77 list of 7 values
07 01 00 01 08 02 7f obj name
01 optional status
01 optional time
62 1e unit
52 03 scaler
69 00 00 00 00 00 00 00 00 value
01

invalid entry
77 list of 7 values
07 01 01 47 02 05 7f obj name
01 optional status
01 optional time
01 optional unit
01 scaler
03 02 76 value
36 <--- error

remaining bytes
22195d6e03607b6f0647065517504a395d0f1a7b6f7237190112595f437272504c201f5e67705a3530201540202a010101631e730076050541733762006200726302017101637b2e001b1b1b1b1a007e01

I don't know what kind of entry this is, but it looks indeed wrong.

[Apollon77]

Honestly ... then the smartmeter is sending an invalid sml :-( And the error is really always at the very same position?

[nknapp]

Yes, always offset 223 and always 0x36. Is there usually any way to fix such an error in the smartmeter? Firmware upgrade?

Is there any strategy to figure out what the remaining bytes might mean?

[Apollon77]

You you can try to contact the energy partner and tell them about that error and let them check how to fix that. Most likely they will exchange the smart meter ... (no idea if firmware updates are even supported for such devices).

WHen I find time I can debug in detail, but the error normally tells that it is invalid data, sorry

[nknapp]

OK, thanks that info and the offer.

But you don't have to debug this just for me. I'm sure you have other things to do as well. I can try to figure out what's wrong. And I can also try to collect impulse data with a simple foto-diode. That won't even need SML.

I'll close this issue. If you really want to work on this, feel free to re-open. But if I am the only one who needs this, I wouldn't want to steal your time.

[nknapp]

Ok, one more thought: All the counter values actually seem to be in the message. It's just the glibberish at the end that causes the parser some pain.

If it would be possible to retrieve that part of the message until the error came up, that would already help a lot. Maybe also other people.

That would be a new feature of course.

[nknapp]

I have tried to decode the message a bit more. In this string, lists are indented, some items are annotated, if I could identify them. Anything after the error is pure assumption of course, but the funny thing is that "7636" and "0655" and some other values of the "glibberish block" part of the public-key that is printed on the smartmeter. They also appear in the same position in the public-key and in the glibberish-block. The size also matches.

1b1b1b1b
01010101
76 list of 6
  05 05 41 73 35
  62 00
  62 00
  72 list of 2
    63 01 01 unsigned 257 
    76 list of 6
      01
      01
      05 01 40 7b 3d
      0b 09 01 49 53 4b 00 04 03 73 7a
      01
      01
  63 3f 76 unsigned 15478
  00
76
  05 05 41 73 36 
  62 00
  62 00
  72 
    63 07 01 
    77
      01
      0b 09 01 49 53 4b 00 04 03 73 7a server-id
      07 01 00 62 0a 7f 7f 
      72
        62 01
        65 02 51 04 58
      76 
        77
          07 01 01 47 02 03 7f 
          01
          01
          01
          01
          04 49 53 4b
          01
        77
          07 01 00 00 00 09 7f
          01
          01
          01
          01
          0b 09 01 49 53 4b 00 04 03 73 7a
          01
        77
          07 01 00 01 08 00 7f
          65 00 00 01 02
          01
          62 1e
          52 03
          69 00 00 00 00 00 00 12 20
          01 
        77 
          07 01 00 01 08 01 7f 
          01
          01
          62 1e 
          52 03
          69 00 00 00 00 00 00 12 20 value 4640 (kWh)
          01
        77 list of 7 values
          07 01 00 01 08 02 7f obj name
          01 optional status
          01 optional time
          62 1e unit
          52 03 scaler
          69 00 00 00 00 00 00 00 00 value
          01
        77 list of 7 values
          07 01 01 47 02 05 7f obj name
          01 optional status
          01 optional time
          01 optional unit
          01 scaler
          03 02 <- last octet string before error
            |  7636 2219 5d6e 0360 7b6f 0647
            |  0655 1750 4a39 5d0f 1a7b 6f72
            |  3719 0112 595f 4372 7250 4c20 
            |  1f5e 6770 5a35 3020 1540 202a public key? some values match
          01 
      01
      01 
  63 1e 73
  00 
  76 
    05 05 41 73 37  
    62 00 
    62 00
    72 
      63 02 01
      71 
    01 
    63 7b 2e 
00 end 
1b1b1b1b
1a007e01

[nknapp]

I believe, if the line

03 02 <- last octet string before error

would be a

83 02

everything would be fine, right?

According to "6.1 Type-Length-Field", the total length would then be 32 hex, which is 50, minus the two TL-fields, is 48, which is the length of the public key.

[Apollon77]

In fact with this length should have a two byte TLField and this is not the case here, but yes the "03" should be something else. I can not calculate the real value right now - but what you wrote could be right - this code parses it: https://github.com/Apollon77/OpenSML/blob/master/lib/SmlBuffer.js#L234-L249 ... somaybe you can work around it for your counter by doing a replace on the hex data before parsing it ;-)

[Apollon77]

Ok, so we have invalid data, nothing the lib can do against