kriszyp / msgpackr

Ultra-fast MessagePack implementation with extension for record and structural cloning / msgpack.org[JavaScript/NodeJS]
MIT License
490 stars 53 forks source link

Decoding to `Map` instead of `Object` for data serialized using `rmp_serde` rust messagepack crate #120

Open twitu opened 1 year ago

twitu commented 1 year ago

I'm sending data from a rust server to a js client. The rust side is using rmp_serde a messagepack crate. And on the JS side is msgpackr. Sending across an object like this gives.

#[derive(Serialize, Deserialize, Debug)]
struct Person {
    name: String,
    best_friend: Option<Box<Person>>,
    age: usize,
    address: Vec<String>,
}

    // Convert bob to bytes and send it
    let data_bob = to_vec_named(&bob).expect("Serialization failed");
          let message = codec.decode(msg.data)
Received: object
Received Object: Map(4) {
  'name' => 'Bob',
  'best_friend' => Map(4) {
    'name' => 'Alice',
    'best_friend' => Map(4) {
      'name' => 'David',
      'best_friend' => null,
      'age' => 27,
      'address' => [Array]
    },
    'age' => 29,
    'address' => [ '567 Birch St' ]
  },
  'age' => 28,
  'address' => [ '456 Oak St' ]
}

This map has to explicitly be converted into a JS object. Ideally the output should be as below. When doing both encoding and decoding using msgpackr outputs a JS object, as is expected.

Received: object
Received Object: {
  name: 'Bob',
  best_friend: {
    name: 'Alice',
    best_friend: { name: 'David', best_friend: null, age: 27, address: [Array] },
    age: 29,
    address: [ '567 Birch St' ]
  },
  age: 28,
  address: [ '456 Oak St' ]
}

Is there some data/configuration missing on either side for messagepack style ser/de? Am I missing something here?

kriszyp commented 1 year ago

The default unpack/decode method should be deserializing as objects, but if you created a Unpackr/Packr instance, you can set useRecords: false or mapsAsObjects: true:

const packr = new Packr({ useRecords: false });
unpackr.decode(data);
twitu commented 1 year ago

Thanks for sharing these options both of them work. useRecords: false and mapsAsObjects: true.

I was trying to do something similar by explicitly creating objects like this below. Do these options do something more efficient? I'm asking because this step will be part in the hot path of the api logic and the most CPU efficient solution will be preferable.

function convertToJsObject(obj) {
  if (obj instanceof Map) {
    const jsObject = {}
    obj.forEach((value, key) => {
      jsObject[key] = convertToJsObject(value)
    })
    return jsObject
  } else if (Array.isArray(obj)) {
    return obj.map((item) => convertToJsObject(item))
  }
  return obj
}

It's also a bit weird that packing and unpacking with msgpackr decodes it to objects just fine. But packing with rmp_serde makes it behave differently.