casper-network / casper-sidecar

Apache License 2.0
2 stars 7 forks source link

Change serialization scheme of Transaction::Version1 variant #326

Open zajko opened 1 week ago

zajko commented 1 week ago

As a maintaner of the node I want to be able to extend the datatypes related to Version1 treansactions without the need of all the clients to be aware of said change. To do that we need to prepare a serialization procedure that is forward-compatible with adding new fields to types and enum variants. The following assumptions have been made:

  1. We want to go with the map Map solution -> simpler, less overhead, simpler code

  2. The first swing at the serialized map -> Map<u16, (u32, bytes[])> (obviously we need one that serializes deterministically)

  3. We focus ONLY on adding fields or enum variants. We assume that if a requirement occurs in the future that changes this assumption that is rationale enough to introduce Version2 transaction

  4. We don't consider any existing protocol for serialization (protobuf et consortes seems like overkill for the task at hand)

  5. All fields are serialized with their respective to_bytes/from_bytes, sth like:

    struct X {
      a: u8
      b: Y,
    }
    (...)
    toBytes()(...) {
     let the_map = (...);
     the_map.insert(A_FIELD_INDEX, (a_size, a.to_bytes()?);
     the_map.insert(B_FIELD_INDEX, (b_size, b.to_bytes()?);
    }
  6. All new fields are OPTION. OPTION fields are serialized as a no-entry in the serialized map. A SOME option is serialized as the underlying type. (Do we want to apply this logic to existing OPTION fields? probably?)

  7. We want to support "fat rust enums" by serializing a TAG which denotes the enum variant and storing the variant data as a struct, sth like:

    enum Z {
    A,
    B {v: u8},
    }
    A_VARIANT_TAG = 0;
    B_VARIANT_TAG = 1;
    TAG_INDEX = 0;
    V_FIELD_INDEX = 1;
    toBytes()(...) {
     let the_map = (...);
     match self {
        A => {
            the_map.insert(TAG_INDEX, (u8::size, A_VARIANT_TAG.to_bytes()?);};
        },
        B{ v } => {
            the_map.insert(TAG_INDEX, (u8::size, B_VARIANT_TAG.to_bytes()?);};
            the_map.insert(V_INDEX, (v.serialized_length(), v.to_bytes()?);};
        },
        (...)
    }
  8. We want to treat tuple enums like structs (maybe we would want to change them in the rust code as well - need to estimate the scope on that), so:

    enum Z {
    X(u8)
    }

    would be treated as (and we think that prefferrably we should adopt this style)

    enum Z {
    X { val: u8}
    }

and would end up with serializing like:

toBytes()(...) {
     let the_map = (...);
     match self {
        X( val ) => {
            the_map.insert(TAG_INDEX, (u8::size, X_VARIANT_TAG.to_bytes()?);};
            the_map.insert(VAL_INDEX, (val.serialized_length(), val.to_bytes()?);};
        },
        (...)
   }