eed3si9n / sjson-new

a typeclass based JSON codec that's backend independent
Apache License 2.0
36 stars 19 forks source link

Fix benchmarks for msgpack #48

Closed jvican closed 7 years ago

jvican commented 7 years ago

Msgpack is in clear disadvantage since it has to translate from the Java API to the Scala API (collections).

This PR fixes:

Two notes:

These benchmarks should be merged with the ones done by @dkomanov: https://medium.com/@dkomanov/scala-serialization-419d175c888a.

The highly regarded serialization benchmark suite in the JVM is http://eishay.github.io/jvm-serializers/prototype-results-page/#.

We have to be doing something wrong, since that above benchmark and Uber's benchmark decides that the best solution for fast serialization and small file size is Zlib Messagepack.

http://highscalability.com/blog/2016/3/21/to-compress-or-not-to-compress-that-was-ubers-question.html

Note: sjson-new creates too much abstraction and it's not as efficient as it could be. Pluggable backends have this deficiency.

jvican commented 7 years ago

Last thing: the results provided by Travis CI should not be used as reliable benchmark results. CPU frequency changes and Travis machines leave much to be desired.

These are my results for gzipped Spray vs MsgPack:

[info] GzipSprayBenchmark.moduleId1SaveToFile      avgt   20  881.388 ± 12.475  ms/op
[info] GzipSprayBenchmark.moduleId2LoadFromFile    avgt   20  630.591 ± 72.937  ms/op
[info] MessagePackBenchmark.moduleId1SaveToFile    avgt   20  373.162 ± 62.374  ms/op
[info] MessagePackBenchmark.moduleId2LoadFromFile  avgt   20  714.574 ± 71.670  ms/op