hyperlane-xyz / hyperlane-monorepo

The home for Hyperlane core contracts, sdk packages, and other infrastructure
https://hyperlane.xyz
Other
307 stars 339 forks source link

Use standardized serializer generation for internal db format #2623

Open mattiekat opened 1 year ago

mattiekat commented 1 year ago

Currently in the code we have a lot of hand rolled serialization code using this idea of the Encode and the Decode trait. While there are a couple of well-defined sterilization formats (such as HyperlaneMessage), there are also a bunch which we just use internally and defining Encode and Decode using either Borsh or Bincode would make a lot of sense.

Some things I noticed will investigating: 1) Borsh has support for exporting the schema, this could be useful for ensuring the data format has not changed since the value was put in the database and catch migration bugs. It would also be a good way for us to handle migrations in the future, by detecting what version was used. 2) We probably want to maintain the Encode/Decode interface so that types which need a specific implementation can still be supported. 3) Encode and Decode should support using the bytes crate to eliminate the unwrap error handling cases. Pretty sure a BufMut would work perfectly in our case.

nambrot commented 7 months ago

@tkporter should we keep this open?