tkporter
commented
1 year ago Asked a long question in Slack seeking some advice:
Hi! I’m hoping to get some advice around encoding / decoding to and from packed bytes.
For context, we require this for 2 reasons: Our protocol relies very heavily upon some data structures being packed in a manner similar to Solidity’s abi.encodePacked, being hashed, and signed by some off chain entities Developers that send messages via Hyperlane do so by encoding messages into blobs of bytes which are sent “over the wire” of our messaging protocol. So they must be able to encode and decode messages pretty easily
For 1, so far we’ve implemented some “manual” packing by allocating bytes on the heap, doing some fancy bitwise operations to pack all our data one word at a time, storing each packed word on the heap with the store word instruction, and then using the k256 instruction to hash. We similarly decode by reading words and doing some bitwise operations.
We saw the new Bytes type which seems like it does something similar in that it helps deal with packed bytes, but it so far doesn’t really help with easily encoding / decoding data— we’d still need to break each variable we want to encode down into u8s via some bitwise operations, and do some similar bitwise operations for decoding. It also doesn’t (yet) support logging the bytes or hashing them.
So we’re at a bit of a crossroads - we think it probably makes sense to move over to the Bytes type because it’s in the stdlib, but are also hesitant to write all those bitwise operations to support encoding / decoding. So a few questions for you guys:
Do you guys have concrete plans for better encoding / decoding support for the Bytes type before mainnet? We noticed the memory copy and memory copy immediate instructions. We’re wondering if it’s possible to leverage these in a nice way for encoding / decoding. For example, to encode a u32 that (iiuc) lives on the stack, to use MCPI to copy the 4 bytes where the u32 lives on the stack into a specific range of 4 bytes in the packed byte data type that lives on the heap. I know as it’s written today this would require changes to the Bytes data type, but an API that allows us to not need to cut a type up into many u8s at a time or do fancy bitwise operations would be much much nicer for us, so we’d consider implementing this into the Bytes type ourselves for now. Not sure if MCP/MCPI work at all for stack types though - also is possible I'm misunderstanding a lot here :slightly_smiling_face: Any general advice would be greatly appreciated!
yorhodes
Atm, we've handcrafted the packed bytes of an encoded message. We should try to work with the Fuel team to be able to use the Bytes type, ideally without requiring ourselves to do a ton of bitwise operations.