search

kaspanet / rusty-kaspa

Kaspa full-node and related libraries in the Rust programming language. This is a stable version at the initial rollout phases.
ISC License
371 stars 118 forks source link

wRPC binary serialization with data structure versioning support #480

Open aspect opened 3 weeks ago

aspect commented 3 weeks ago

This PR implements support for "wRPC serialization versioning".

This is accomplished using a set of primitives and macros added to workflow-rs (workflow-serializer crate) that implements Serializer trait and a helper Serializable<T>(T) struct (https://github.com/workflow-rs/workflow-rs/tree/serializer/serializer/src).

wRPC calls require BorshSerialize and BorshDeserialize. These traits have been removed form RK RPC data structures. Instead they now implement Serializer trait to perform binary serialization. The Serializable<T>(T) where T: Serializer struct implements BorshSerialize and BorshDeserialize and is #[repr(transparent)], effectively converting data structures that implement Serializer trait to BorshSerialize and BorshDeserialize invocations.

Each RPC (and related) data structure that opts-in for versioning now manually serializes it's fields, where the first field is always a version number (currently u32). By checking the version number on deserialization, structures can opt-in to deserialize old or new binary layouts.

This is a "future compatible" implementation that allows newer clients/nodes to understand older requests, but will not allow older clients/nodes to understand newer requests. (each structure can implement forward compatibility internally if desired by pre-buffering and storing the size of the binary payload and ignoring (while skipping) additional data; this is not really a common approach as it poses other issues where field types can change, breaking serialization).

By augmenting Kaspa wRPC client and server interfaces with these traits, the constraints on these interfaces change requiring all RPC data structures to implement the Serializer trait, which is now implemented for each such struct.

The implementation of serialization is done beside each RPC data structure. A new rpc/core/src/test.rs file implements serialization unit tests designed to catch basic data misalignments.

IMPORTANT: When this PR is merged, we will need to make a new node release as well as the corresponding WASM SDK and KNG releases as this will break wRPC Borsh encoding compatibility (this PR addresses that very problem, allowing us to perform protocol upgrades in the future without breaking such compatibility).