near / borsh-js

TypeScript/JavaScript implementation of Binary Object Representation Serializer for Hashing
Apache License 2.0
112 stars 38 forks source link

How to serialize Enum? #21

Closed max-block closed 10 months ago

max-block commented 3 years ago

I haven't found any example to to serialize an enum. Can you please give an example to to do it. For example, here is a Rust enum:

 #<span class="error">[derive(BorshSerialize, BorshDeserialize, PartialEq, Debug)]</span>
 enum CalcInstruction {
 Init,
 Plus 

 { value: i32 } 

,
 Minus 

 { value: i32, comment: String } 

,
 }
jsoneaday commented 3 years ago

Similarly I would like to serialize an array of some type but its fails with error Class array is missing in schema

max-block commented 3 years ago
jsoneaday commented 3 years ago
  • serialization of a vector :)

In javascript? What would be a vector?

AuroraLantean commented 3 years ago

You can convert an enum array into number array, then new Uint8Array(enumNumber_array);

ilmoi commented 3 years ago

good news.. I think I figured it out. bad news.. it's not pretty. And probably not optimal.

But here it goes.

Step 1: serialize the enum variant you want to send (on its own). In my case it was a struct:

pub enum VestingInstruction {
    // more variants here

    // this is the one I want to send
    Empty {
        number: u32,
    },
}

so I serialized it as a class:

  class Empty {
    number = 0;

    constructor(fields) {
      if (fields) {
        this.number = fields.number;
      }
    }
  }

  // we'll need the size later
  const empty_schema = new Map([[Empty, {kind: 'struct', fields: [['number', 'u32']]}]]);
  const empty_size = borsh.serialize(empty_schema, new Empty()).length;

  // this is the byte array containing the serialized "Empty" variant with a number 5 in it
  const empty_serialized = borsh.serialize(empty_schema, new Empty({number: 5}));

Step 2: create a class with your instructions. Like the name suggests we'll be passing in serialized versions of our enum variants later on.

class Ix {
  init = 'serialized_init';
  create = 'serialized_create';
  unlock = 'serialized_unlock';
  changed = 'serialized_changed';
  empty = 'serialized_empty';

  constructor(fields) {
    if (fields) {
      this.serialized_init = fields.serialized_init;
      this.serialized_create = fields.serialized_create;
      this.serialized_unlock = fields.serialized_unlock;
      this.serialized_changed = fields.serialized_changed;
      this.serialized_empty = fields.serialized_empty;
    }
  }
}

Step 3: serialize the instruction.

// this is where our previously serialized data (from step 1) goes
// we only need to populate the one we care about
const ixx = new Ix({serialized_empty: empty_serialized});

// we need to specify the size for each serialized instruction.  We derive it above in step 1.
const values = [
  ['serialized_init', [init_size]],
  ['serialized_create', [create_size]],
  ['serialized_unlock', [unlock_size]],
  ['serialized_changed', [changed_size]],
  ['serialized_empty', [empty_size]]
];

// this is where we finally build an enum. 
// -Field = which variant we want. 
// -Values = mapping of names to array sizes.
const schema = new Map([[Ix, {kind: 'enum', field: 'empty', values: values}]]);

// final step - actually serialize it
const data = borsh.serialize(schema, ixx);

The above successfully works with this on the rust end:

impl VestingInstruction {
    pub fn unpack(input: &[u8]) -> Result<Self, ProgramError> {
        let result: Self = Self::try_from_slice(input).unwrap();
        Ok(result)
    }

Like I said.. it ain't pretty. I basically reverse engineered from the library code. If one of the maintainers could comment with a better solution that would be awesome.

ilmoi commented 3 years ago

A slightly hackier version of the above looks like this:

  class Ix {
    data = 'serialized_data';

    constructor(fields) {
      if (fields) {
        this.serialized_data = fields.serialized_data;
      }
    }
  }

  const ixx = new Ix({serialized_data: empty_serialized});

  const values = [
    [],
    [],
    [],
    [],
    ['serialized_data', [empty_serialized.length]],
  ];

  const schema = new Map([[Ix, {kind: 'enum', field: 'data', values: values}]]);
  const data = borsh.serialize(schema, ixx);

This also works. The important part here is the values array - we must pass in our serialized enum variant as i'th item into the array, where i = matches enum in rust.

So eg in my case I had 5 variants in my rust enum and I wanted to trigger the 5th. So I pased in the data as the 5th item into the array on js end.

This is because we basically need borsh to insert "4" (5th item starting from 0) as the first byte into the byte array it sends to the program.

Not sure if better or worse approach.

ilmoi commented 3 years ago

Coming back to this some time later... there's a far easier solution that I can now see: 1)Serialize your instruction without the enum part 2)Manually prepend the enum part

  // 1)
  const empty_schema = new Map([[Empty, {kind: 'struct', fields: [['number', 'u32']]}]]);
  const empty_size = borsh.serialize(empty_schema, new Empty()).length;
  const empty_serialized = borsh.serialize(empty_schema, new Empty({number: 5}));

  // 2) In my case the instruction is the 5th in order
  const data = Buffer.from(Uint8Array.of(4, ...empty_serialized))

Might not work for all use-cases, but for instructions this seems superiod to what I suggested above.

marcus-pousette commented 2 years ago

I made a draft PR https://github.com/near/borsh-js/pull/39, that could make this a bit cleaner. What we could do with it

@Variant(4)
class Empty {
    @Field({ type: 'i32' })
    number = 0;
    constructor(fields) {
      if (fields) {
        this.number = fields.number;
      }
    }
 }
const generatedSchemas = generateSchema([Empty])
const buf = serialize(generatedSchemas,  new Empty({number: 5}));

Variant(4) means instruction is the 5th in order

dgabriele commented 2 years ago

It would be great if PR39 got reviewed and merged soon. It should be much more straight-forward to serialize enums. It would be much appreciated!

marcus-pousette commented 2 years ago

It would be great if PR39 got reviewed and merged soon. It should be much more straight-forward to serialize enums. It would be much appreciated!

There has been no activity by the code maintainers since my last post. As I need these changes for my own project (which is "enum heavy") I started to maintain my own fork in the mean time.

https://github.com/dao-xyz/borsh-ts

I also added the support for enum deserialization. See this test.

class Super {}

@variant(0)
class Enum0 extends Super {
    @field({ type: "u8" })
    public a: number;

    constructor(a: number) {
      super();
      this.a = a;
    }
}

@variant(1)
class Enum1 extends Super {
    @field({ type: "u8" })
    public b: number;

    constructor(b: number) {
        super();
        this.b = b;
    }
}

class TestStruct {
    @field({ type: Super })
    public enum: Super;

    constructor(value: Super) {
        this.enum = value;
    }
}

const instance = new TestStruct(new Enum1(4));
const serialized = serialize(instance);
expect(serialized).toEqual(Buffer.from([1, 4]));

const deserialized = deserialize(
    Buffer.from(serialized),
    TestStruct,
);

expect(deserialized.enum).toBeInstanceOf(Enum1);
expect((deserialized.enum as Enum1).b).toEqual(4);

Feel free to use it if you find it useful.

dgabriele commented 2 years ago

It would be great if PR39 got reviewed and merged soon. It should be much more straight-forward to serialize enums. It would be much appreciated!

There has been no activity by the code maintainers since my last post. As I need these changes for my own project (which is "enum heavy") I started to maintain my own fork in the mean time.

Many thanks! I hope the PR gets merged soon :)

gagdiez commented 10 months ago

should be fixed now, please open again if not