More MsgPack optimizations

[kerams]

Just playing around with some cool APIs, which I don't get to do at work :))

Type	Method	Job	UnrollFactor	Mean	Error	StdDev	Median	Gen 0	Gen 1	Gen 2	Allocated
Int64ArraySerializationBefore	MsgPack	DefaultJob	16	15.21 ms	0.112 ms	0.104 ms	15.22 ms	359.3750	62.5000	62.5000	4.29 MB
Int64ArraySerializationAfter	MsgPack	DefaultJob	16	11.50 ms	0.226 ms	0.211 ms	11.50 ms	359.3750	62.5000	62.5000	4.29 MB
IntMaybeMapSerializationBefore	MsgPack	DefaultJob	16	11.973 ms	0.0418 ms	0.0371 ms	11.959 ms	859.3750	46.8750	-	7.29 MB
IntMaybeMapSerializationAfter	MsgPack	DefaultJob	16	6.750 ms	0.0469 ms	0.0438 ms	6.756 ms	671.8750	23.4375	7.8125	5.69 MB
RecursiveRecordSerializationBefore	MsgPack	DefaultJob	16	399.717 ms	1.3956 ms	1.3055 ms	399.781 ms	14000.0000	-	-	128.5 MB
RecursiveRecordSerializationAfter	MsgPack	DefaultJob	16	142.02 ms	0.676 ms	0.632 ms	-	6000.0000	-	-	64.43 MB

[kerams]

Actually, I have one more improvement up my sleeve.

[kerams]

Think I'm done for now. These are the final results:

Type	Method	Job	UnrollFactor	Mean	Error	StdDev	Median	Gen 0	Gen 1	Gen 2	Allocated
Int64ArrayDeserialization	Json	DefaultJob	16	39.092 ms	0.5254 ms	0.4915 ms	38.963 ms	2000.0000	153.8462	76.9231	18.03 MB
Int64ArrayDeserialization	MsgPack	Job-OKQKJW	1	9.849 ms	0.1957 ms	0.5645 ms	9.987 ms	-	-	-	8.39 MB
Int64ArraySerialization	Json	DefaultJob	16	58.784 ms	0.4902 ms	0.4093 ms	58.751 ms	5000.0000	1000.0000	-	47.71 MB
Int64ArraySerialization	MsgPack	DefaultJob	16	11.663 ms	0.0909 ms	0.0806 ms	11.633 ms	359.3750	62.5000	62.5000	4.29 MB
IntMaybeMapDeserialization	Json	DefaultJob	16	727.853 ms	12.1020 ms	11.3202 ms	721.926 ms	35000.0000	9000.0000	-	286.17 MB
IntMaybeMapDeserialization	MsgPack	Job-OKQKJW	1	38.226 ms	0.7611 ms	2.2082 ms	39.057 ms	2000.0000	1000.0000	-	22.67 MB
IntMaybeMapSerialization	Json	DefaultJob	16	236.839 ms	0.5522 ms	0.4611 ms	236.891 ms	9000.0000	1000.0000	-	76.1 MB
IntMaybeMapSerialization	MsgPack	DefaultJob	16	5.771 ms	0.0694 ms	0.0649 ms	5.776 ms	609.3750	39.0625	7.8125	5.23 MB
RecursiveRecordDeserialization	Json	DefaultJob	16	1,109.944 ms	16.6002 ms	15.5279 ms	1,104.495 ms	55000.0000	12000.0000	-	438.84 MB
RecursiveRecordDeserialization	MsgPack	Job-OKQKJW	1	453.591 ms	5.3662 ms	5.0195 ms	452.036 ms	19000.0000	5000.0000	-	157.21 MB
RecursiveRecordSerialization	Json	DefaultJob	16	594.457 ms	11.5427 ms	11.3364 ms	591.723 ms	30000.0000	9000.0000	1000.0000	271.83 MB
RecursiveRecordSerialization	MsgPack	DefaultJob	16	107.279 ms	0.6508 ms	0.5769 ms	107.212 ms	6000.0000	-	-	64.43 MB

[Zaid-Ajaj]

These numbers are amazing @kerams 🤯 I would lie to you if I say that I really understand what is going on there. Though I see that the tests added are only running within a dotnet host with Expecto and not with Node and Fable.Mocha correct? If possible, it would be great to also add some integration tests with a couple of test samples like the ones you are adding for MsgPack.Tests. Ignore me of course if you already covered those cases with the existing integration tests and I didn't notice :smile:

[kerams]

I believe many tests overlap (no problem with that) and are spread across these files/projects:

• Fable.Remoting.MsgPack.Tests - runs on .NET - checks that serialization -> deserialization produces an object identical to the input
• Fable.Remoting.Giraffe.Tests/MiddlewareTests.fs - runs on .NET - integration test for both Giraffe and .NET client. Whatever test works in MsgPack.Test is pretty much guaranteed to work here too (because both ends are .NET again)
• ClientV2/App.fs - runs with Mocha
  • test list binaryServerTests - integration tests for the JS client and Suave server (UITests.fsproj). What's serialized in Suave has to be correctly deserialized in JS
  • test list msgPackTests - the JS only counterpart to MsgPack.Tests, important because of major differences in (de)serialization code between .NET and JS (conditional compilation)

Apart from fixing some units of measure serialization issues (tests for which have been added to App.fs), I have only touched the .NET serialization parts in this PR. There are some extra test cases in MsgPack.Tests, but you're right in that it wouldn't hurt to have them covered in binaryServerTests too. I'll add it tomorrow.

Feel free to ask if you have specific questions about the implementation.

[kerams]

Note to self: next time run the tests you have just written before pushing to save yourself the embarrassment.

Had to remove 2 tests though:

• Request with Some None is serialized to JSON as null, which tryCreateArgsFromJson correctly parses as just None
• Request with ValueSome "yupp" is serialized to JSON as "yupp", but tryCreateArgsFromJson fails (not sure if the problem is specific to this type or struct unions in general, though I doubt anyone would actually use these in the request anyway)

[Zaid-Ajaj]

Feel free to ask if you have specific questions about the implementation

Thanks a lot :heart: I will go and learn the MsgPack specs first, that will makes things easier to understand

Had to remove 2 tests though:

Yeah I will make separate issues out of those. Like you said, probably no one uses either but it is good to have them tracked for the sake of completeness :100:

[Zaid-Ajaj]

Published a batch of all the libraries (except for Fable.Remoting.Json ofc) which include these optimizations :smile: