Out-of-band management protocol

[si14]

I'm (well, will be) shipping devices and host software to customers. Both the firmware and the host software are being actively developed. Here are two scenarios that worry me a lot:

• Protocol changes. I think avoiding anything complex with forward/backward compatibility in Postcard (RPC) is a good call, but imagine that someone got a new host software version together with a device still flashed with an old firmware. How should they update it? It's a chicken-and-egg problem, since the RPC protocol can't be used to initiate firmware upgrade (the version mismatch will be detected and sensibly cause an error), but for the RPC protocol to be useful again the firmware needs to be updated.
• Bugs. This is a nightmare scenario, but what if I wrote a bugged firmware upgrade RPC handler? This would effectively brick the device, requiring a debug probe to unbrick it. "Just don't write bugs" is fair, but what if Postcard RPC gave me a get-out-of-jail-free card?

In both cases, it'd be really useful to have a simple out-of-band management protocol, the simpler the better. The details are microcontroller-dependent, but at least for RP2040 a single SDK call (reboot to stage 1 bootloader) would be sufficient and really hard to get wrong. It can be completely out-of-band (a separate USB endpoint? using the control endpoint?), or it can just be a magic value in the existing protocol.

There might be more than one layer to this protocol. I can imagine a two-layer protocol, with "reboot into firmware update mode" being the simplest, dumbest, least likely to break layer, and something like "please return the current version/the last panic message/etc" being a bit more complex, but still more reliable than user code.

Either way, I think having something like this in Postcard RPC would be extremely useful. Let me know what you think!

[jamesmunns]

Note that the way that postcard-rpc is written, the "breaking changes" are granular to each endpoint, NOT the whole version. If you only change one API's types, then only that API is broken.

There may still be breaking "base protocol" changes until postcard-rpc 1.0. This is something that you'll have to manage if you begin shipping devices before then.

I do have some plans to offer .well-known endpoints, for example "give me all the endpoints/topics you support". This could be used host side for version detection. My recommendation would be to have the host "understand" many versions worth of endpoints, and each device only supports its own.

Similarly, you can write your own stable "get version" or "start bootload" APIs.

what if I wrote a bugged firmware upgrade RPC handler

I'd strongly suggest doing integration testing.

There might be more than one layer to this protocol.

At the moment, I'd like to keep postcard-rpc very dumb and general at it's core. I'd welcome "by convention" patterns/cookbooks, but I'd like to separate "what postcard-rpc does" from project's actual application/business logic.