jamesmunns / postcard

A no_std + serde compatible message library for Rust
Apache License 2.0
874 stars 87 forks source link

Feature request: partial backwards compatibility #159

Open tomhampshire opened 3 months ago

tomhampshire commented 3 months ago

I was thinking - it would be really nice to have some backwards compatibility between schemas, especially when iterating quickly on new features/comms. There's likely some ongoing or existing work/discussions regarding this (it may already exist...), but I have not been able to find a conclusive answer (docs/wire spec/etc), so I wanted to raise it here to ask.

If certain structs were frozen, it would seem that the main thing that holds back any backwards compatibility is tagged union's variant changing their discriminant during (de)serialization. Is there a way to fix this discriminant, so that I know that particular component of the comms is going to remain stable?

For example, when going from:

#[derive(Serialize, Deserialize)]
pub enum MessageRequest {
    SetDatetime(i64),
    GetDatetime,
    ToggleLed,
}

to

#[derive(Serialize, Deserialize)]
pub enum MessageRequest {
    SetDatetime(i64),
    GetDatetime,
    ToggleLed,
    Shutdown
}

It would be great to know that the existing comms would work as before, unless you used the MessageRequest::Shutdown enum (for which the error could be handled). My naive view would be, if you could guarantee the discriminant for a variant (either by ordering, or by assigning a fixed "tag"), this partial backwards compatibility would be assured.

jamesmunns commented 2 months ago

In general, you can add enum variants and add fields to the end of a struct, BUT the issue is that this all generally breaks down if these types are nested. If you add a field to a struct that is in the middle of another struct, everything will go wrong.

I don't believe this is fixable with postcard's "non self describing format" design choice. The wire format is specified enough that you could potentially add a layer on top of postcard that handles some kind of schema flexibility, but it isn't possible out of the box, as far as I am aware.