Python Simple Management Protocol library (AKA mcumgr)

[JPHutchins]

After some struggles with USB DFU, and discussions on the Zephyr Discord server, I've decided to fully invest in the Simple Management Protocol as the means of providing DFU over USB, BLE and anything else that comes along. The protocol is maintained here and it is well defined. The protocol can be extended easily and custom SMP handlers on Zephyr firmware (the SMP "server") can be defined in ~40 lines of code due to good documentation and CBOR parsing.

It would be helpful to have a permissively-licensed SMP library that can be used to 1) build customers DFU solutions or 2) use as a reference for customers DFU solutions. A list of projects is maintained here:

https://docs.zephyrproject.org/latest/services/device_mgmt/mcumgr.html#tools-libraries

For mobile applications, nRF Connect Device Manager fulfills this role.

mcumgr-web opens some interesting possibilities for companies moving to PWAs and similar, but it needs to be extended for serial, implement more groups, and get test coverage - this is DFU after all. Importantly, it's not much use from within the Zephyr development environment.

On the PC side, I would like a universally compatible library and CLI that implements SMP for all transports.

I have worked with mcumgr-client, adding a GitHub release workflow for Windows/Linux/MacOS distribution. But, mcumgr-client is a minimal implementation without test coverage, is not a library, and makes some wrong assumptions about SMP.

At Intercreate we have taken a 3-tiered approach to implementing SMP in Python:

• smp - Apache-2.0 library for SMP de/serialization
• smpclient - Apache-2.0 library that implements the serial and BLE transports
• smpmgr - universally portable CLI for interacting with SMP servers

All of these are available from PyPI. I recommend pipx or the portable if you want to try smpmgr as an alternative to mcumgr.

It's hard to manage and test every feature and use case for SMP, so I'm hoping to gather feedback regarding requirements for this kind of infrastructure. In the long run, I believe that it may be useful to have a reference SMP client implementation available from within a Zephyr development environment to smooth the troubleshooting process of DFU, security, and other management features.

Cheers, JP

[nordicjm]

CC @carlescufi

[nordicjm]

I have worked with mcumgr-client

If you mean the go tool then that tool should just not be used at all, it has long been broken and never complied with the protocol properly.

There has been a longstanding idea to have a python client but it's just never happened due to other higher priority tasks so this looks like a good idea, I don't use python so I can't comment on the code however.

Important things are that it should support SMP version 1 and 2, should support error code lookups, should use the protocol design that you have linked, and ideally testcases for these. For UART it should also be tested that the encapsulation and decapsulation are working

[JPHutchins]

If you mean the go tool

It’s this great OSS Rust tool by Vouch: https://github.com/vouch-opensource/mcumgr-client

SMP version 1 and 2, should support error code lookups

Agreed!

For UART it should also be tested that the encapsulation and decapsulation are working

Agreed, it has good automated coverage. Because it’s my primary use case it has gotten a lot of time on the bench as well.

[nordicjm]

Would be great to have this then as the primary client, all documentation for the go client could then be removed and replaced with instructions (or links to external documentation, though in-tree would be better for pdf documentation users) on how to set this tool up and use it.

@ddavidebor might be of interest to you

[nordicjm]

I showed this to @carlescufi briefly and one suggestion he made was about the possibility of using a different python bluetooth library, I believe bumble is the one he was talking about https://google.github.io/bumble/ as it's more up to date and natively supports using zephyr-based bluetooth devices

[ddavidebor]

I love this, the tool looks great, and it's nicely packaged in a small executable. Looks like a fantastic replacement that solves a long standing issue.

[JPHutchins]

with instructions (or links to external documentation, though in-tree would be better for pdf documentation users) on how to set this tool up and use it.

I would be happy to do so! Off the top of my head, there are a few milestones to meet first:

1. Add BLE transport to smpmgr
2. More testing on a wider range of devices and use cases
3. "stable" interface - there are breaking changes needed for smp, smpclient, and smpmgr that I know of. There are probably also fixes that will be breaking changes that we don't know of.
4. API documentation for smp and smpclient

For Zephyr users, the simplest means of installation would be into the Zephyr venv - does that sound practical? I suppose it has the downside of duplicating the smpmgr tool with every Zephyr environment. A pipx install adds smpmgr to PATH which is nice because it's then available from every terminal.

I showed this to @carlescufi briefly and one suggestion he made was about the possibility of using a different python bluetooth library, I believe bumble is the one he was talking about https://google.github.io/bumble/ as it's more up to date and natively supports using zephyr-based bluetooth devices

This shouldn't be too difficult to investigate! They have a brave approach... removing the OS layer like Windows BLE driver & BlueZ 💀. MS has made incremental progress but the drivers still have weird issues. It's 2024.

Looks like Bumble setup across OS varies and is certainly invasive, so I suspect that we may end up using Bleak for "end users" - clients of Zephyr FW devs that are not necessarily programmers - and Bumble for Zephyr developers that have dedicated USB BLE dongles.

I love this, the tool looks great, and it's nicely packaged in a small executable. Looks like a fantastic replacement that solves a long standing issue.

Great! LMK if you have any questions and feel free to open bug tickets directly on any of the repos.