How to include parsing of SML bodies

[torfmaster]

Hi,

while implementing https://github.com/torfmaster/hackdose-server I stumbled upon your project. In my project I implemented (rather naively) extracting complete SML messages to control smart plugs based on data coming from my smart meter. My focus was rather on parsing the body to extract data about power consumption than parsing the whole message (e.g. the checksum) which resulted in writing https://github.com/torfmaster/hackdose-sml-parser, a Parser Expression Grammar based parser which does e.g.

• splitting into message types (GetOpenResponse, GetListResponse)
• extracting values from SmlValueLists by Obis numbers

I see that you also have spent quite an effort in doing the encoding part and I wonder whether we could combine the efforts somehow?

I'd be interested in your opinion!

[fkohlgrueber]

Hi,

I just had a quick look at your implementation, very interesting. I haven't used the peg crate yet. The Hackdose project sounds pretty cool. I thought about something similar but never actually started working on it.

Regarding sml-rs: I haven't worked on the project in the last couple of months, but I still intend to continue working on it. I have written a parser using nom that I haven't published yet as it still needs cleanup. I'm not sure which approach works better, but I'd be happy to collaborate on this project. How about this: I'll try to publish my parser soon (say within a week) such that we can discuss which implementation we want to use in sml-rs. Sounds like a plan?

Another thing: Your project is targeted at RPi or similar devices, right? My parser currently uses allocations and I was still figuring out how best to make it work without an allocator. Eventually, I'd like to use sml-rs on microcontrollers like the ESP32-C3 and there, an allocation-free parser would be nice. I'm guessing that you don't need that feature if you're running on RPi, right?

[torfmaster]

Sounds like a plan?

Definitely, it seems like we're targeting different layers of SML (you are targeting transport, I am targeting application layer) so a combination would make sense.

My parser currently uses allocations and I was still figuring out how best to make it work without an allocator.

Some thoughts

• I am targeting the Mango PI just as it has support for most of the hardware I have. Implementing a non_std drivers for WiFi plugs seemed out of scope for my project
• However, targeting microcontrollers (including alloc) like HighFive or STM32 or makes sense for example if you using GPIO to control opto-isolators
• I think having an allocator is completely acceptable if you don't have real time requirements (which the 9600 Bauds of SML suggest) and you are not sending your system to Mars. Done right your software can run for ages without suffering from heap fragmentation. If you are targeting ESP, then you will anyway have an allocator in the WiFi module firmware which is potentially a weaker spot. Not using alloc is super painful as the payload itself can explode in length even with length and capacity limitations (formally you can put lists inside lists inside lists) so you would have to deal with dynamic length payload on a machine which is not able to handle it (or do const generics magic to model your limits).
• handling asynchronous events is hard on microcontrollers. At least for my project I refrained from doing this in the first place because even though there a quite some async libraries for microcontrollers around none of them is as convenient as tokio for now (aside from missing hardware support anyways)

[fkohlgrueber]

Definitely, it seems like we're targeting different layers of SML (you are targeting transport, I am targeting application layer) so a combination would make sense.

Not really. I've been focusing my efforts on the transport layer first, but the end goal is to cover the application layer as well. The "old" branch contains an implementation of the application layer I did a while ago.

Regarding allocations: I agree with you that for most projects, requiring an allocator will be totally fine. I'd like to have the library working without dynamic memory allocations anyway eventually, but that's not a priority. It wont't hurt users that will be using alloc (in the same way as the current implementation of the transport layer is usable with and without an allocator). Besides, it's a fun challenge to design a library for no_alloc and might make it work in even more constrained environments. But as I said, no_alloc is not a priority and more of a nice to have feature.

[felixwrt]

I've just released a new version of sml-rs that includes parsing of SML messages. I think that the library should now work for your use-case in hackdose-server. Feel free to check it out and I'm looking forward to hear what you think about it!

[felixwrt]

Closing this as message parsing is now part of the library.