gagdiez / serial-as

Building a generalized encoder for assembly script
MIT License
13 stars 10 forks source link

MsgPack Support #28

Open dOrgJelli opened 2 years ago

dOrgJelli commented 2 years ago

First off, I really love this implementation, and how it leverages templates to automate code generation :heart:

For the Polywrap project (site, repo), we use MsgPack for encoding/decoding the data we send in & out of WebAssembly. MsgPack is most suitable to our use-case because of the following characteristics:

Currently we manually generate the (de)serialization logic for wasm module types in AssemblyScript (example type + (de)serialization logic).

If this library were to add support for generic MsgPack serialization, we could replace our manual code-gen with this compiler-driven meta-programming based approach.

Implementation Thoughts

After looking at both the JSON & Borsh implementations in this repo, I think MsgPack would look like a hybrid of both approaches.

Deserialization

Since we can't assume member order (like we can with Borsh), we must perform a middle-step (like in JSON) where we first parse the ArrayBuffer being provided. I'm not too sure what metadata this middle-step would store to aid in the final deserialization. Potentially it could store "buffer slice metadata" enabling the easy lookup of types + properties within the blob.

Serialization

This should be very straight forward, and not much different from Borsh's implementation if I'm not mistaken.

Reference MsgPack Implementation

https://github.com/polywrap/monorepo/tree/prealpha/packages/wasm/as/assembly/msgpack

dOrgJelli commented 2 years ago

Additional difference I found in implementations: we made use of a Nullable<T> class to allow us to do things like value: Nullable<bool> since AssemblyScript does not support value: bool | null.

I opened up another issue here to add support for "nullable" scalar types: https://github.com/gagdiez/serial-as/issues/29

gagdiez commented 2 years ago

Started a new branch: https://github.com/gagdiez/serial-as/tree/feat-msgpack

I am sure that with your help we will be able to tackle this promptly.

JairusSW commented 1 year ago

@gagdiez, MessagePack is already implemented 😄 https://github.com/wapc/as-msgpack