doctorn / obake

Versioned data-structures for Rust
Apache License 2.0
203 stars 8 forks source link

Dynamically load correct version? #12

Open jkorinth opened 4 months ago

jkorinth commented 4 months ago

My serialized structs have a header with their version; I'd like to load the exact version dynamically while deserializing, then try to convert to the latest version of the struct. The MyStruct! macros seem to only accept string literals. Is there any way to do this?

doctorn commented 4 months ago

Have you tried deserializing as obake::AnyVersion<MyStruct>?

jkorinth commented 4 months ago

Maybe I misunderstood, but isn't AnyVersion<MyStruct> just a "symbolic link" to the latest version? Meaning, it would always try and parse with the latest version of MyStruct? What I meant was that I may need to deserialize a struct from storage that was serialized by an older definition. In my case there may be incompatible changes of the struct definition, e.g., also removal or reordering of fields.

jkorinth commented 4 months ago

What I tried was something like this:

fn deser(s: &str) -> Result<AnyVersion<MyStruct>> {
    match s {
        "0.1.0" => <MyStruct!["0.1.0"]>::unpack(...),
        ...
    }
}

But obviously this needs to be adapted every time a new version comes around and is not particularly elegant. 😁

doctorn commented 3 months ago

AnyVerson<MyStruct> is an enum of all possible versions. An example of how this is used in practice can be seen here.

jkorinth commented 3 months ago

Not sure I understand. I'm using packed_struct, but without serde because the deserializing end is no_std and no_alloc and cannot use it. So, instead I have to read two blobs, a header with version + checksum, and another &[u8] for the actual data, both of which must conform to an exact binary layout (hence packed_struct). What I'd need is a way to call unpack on the version of the struct that matches header blob info. I don't think AnyVersion will help me, or am I missing something? I'll try an make a MWE tomorrow to better show what I mean.

jkorinth commented 3 months ago

Hope this shows the issue a bit better:

use packed_struct::prelude::*;

#[obake::versioned]
#[obake(version("0.1.0"))]
#[obake(version("1.0.0"))]
#[derive(Copy, Clone, Debug, Default, PackedStruct)]
#[packed_struct(endian = "msb")]
pub struct Data {
    #[obake(cfg("<1.0"))]
    #[packed_field]
    val1: i32,
    #[obake(cfg(">=1.0"))]
    val1: u32,
}

impl From<Data!["0.1.0"]> for Data!["1.0.0"] {
    fn from(d: Data!["0.1.0"]) -> Self {
        Self {
            val1: if d.val1 >= 0 { d.val1 as u32 } else { 0_u32 },
        }
    }
}

fn main() {
    // an older sw stored this version of Data:
    let mut old_data = <Data!["0.1.0"]>::default();
    old_data.val1 = -1;
    println!("old_data = {:?}", old_data);
    // a newer sw reads out with the current def
    // obviously, this will fail (val1 = -1 reinterpreted as u32):
    let new_data = Data::unpack(&old_data.pack().unwrap()).unwrap();
    println!("new_data = {:?}", new_data);

    // What I'm trying to solve is this:
    // I've got the actual version from storage (dynamic)
    let _vers = "0.1.0";
    // now I need to call `unpack` on the right Data def:
    // let b = <Data![_vers]>::unpack( ... );
}