[question] problem encoding types

[endel]

Hey there,

I'm afraid there are some differences between the serialization of this library and Node's msgpack-lite

Still trying to debug to find where the difference is, I'd appreciate if you can give me any hints!

Here's the differences when trying to encode this object:

{ players: {}, messages: [] }

JavaScript:

[130, 167, 112, 108, 97, 121, 101, 114, 115, 128, 168, 109, 101, 115, 115, 97, 103, 101, 115, 144]

C#

// second and third bytes differs from JavaScript
[130, 217, 7, 112, 108, 97, 121, 101, 114, 115, 128, 217, 8, 109, 101, 115, 115, 97, 103, 101, 115, 144]

Still, in JavaScript, I have the same output when trying to decode both byte arrays:

> msgpack.decode([130, 167, 112, 108, 97, 121, 101, 114, 115, 128, 168, 109, 101, 115, 115, 97, 103, 101, 115, 144])
{ players: {}, messages: [] }

> msgpack.decode([130, 217, 7, 112, 108, 97, 121, 101, 114, 115, 128, 217, 8, 109, 101, 115, 115, 97, 103, 101, 115, 144])
{ players: {}, messages: [] }

My guess is that C# is serializing empty objects as a different type.

This issue looks harmless on the surface but unfortunately, it's preventing me from applying byte patches using FossilDelta due to this byte difference.

Looking forward to hearing your thoughts on this! Cheers

[deniszykov]

C# tokens [FixMapStart, Str8 ...

JavaScript: [FixMapStart, FixStrStart...

Looks line msgpack-lite choose better type for encoding this string. I will address this problem soon. But I think this is not the last place where a binary representation of same data could be different. This format was not designed for data storage and patch applying, it was created for messages passing over network. BSON is better for storage, traversal and patching.

[endel]

Thanks for your swift reply @deniszykov!

I've checked the byte size of that same object ({ players: {}, messages: [] }) and it is quite larger using BSON. 34 bytes using BSON and 20 using msgpack. I'd like to keep it as small as possible. :/

If you can give me some direction where to look at in your C# implementation I'd be glad to help.

Cheers! Have a nice weekend :)

[deniszykov]

Hi, I have fixed string serialization issue in commit cfd4017e373527e60cbad99f4912f787981ffbd3. And submitted Unity asset in store (it would take up to one week to update).

Unfortunately I do not have a serializer in the BSON format for Unity.

[endel]

Thanks a lot @deniszykov! Everything looks fine now!

[deniszykov]

I have fixed bug in d72d1cb6540bdba17c51dfa1422e12bef1e5e4aa. It is related to small length strings serialization in MessagePack (rare case with non-latin letters).