improve performance somehow

[antonkulaga]

@ocronos made a performance test where prickle has terrible performance. Something should be done about it

Decoding Seq[Book] with UUIDs
=============================
Library    ops/s      size       %          size.gz    %         
BooPickle  9245       304        100%       304        100%      
Prickle    1974       1117       367%       1117       367%      
uPickle    8425       934        307%       934        307%      

Decoding Seq[Book] with numerical IDs
=====================================
Library    ops/s      size       %          size.gz    %         
BooPickle  18375      192        100%       192        100%      
Prickle    1979       861        448%       861        448%      
uPickle    8545       678        353%       678        353%      

Decoding Seq[Book] with random IDs
==================================
Library    ops/s      size       %          size.gz    %         
BooPickle  11923      264        100%       264        100%      
Prickle    1960       917        347%       917        347%      
uPickle    7957       734        278%       734        278%

[benhutchison]

Keep in mind, BooPickle is a son-of-Prickle anyway, clearly derived from the Prickle sources. It innovates via:

• a more efficient binary encoding. Json will never compete against binary, but Prickle could improve somewhat by using the faster json parsers used by uPickle, at the cost of slightly more complexity.
• identity equality rather than value equality when checking if objects are already pickled. Prickle could borrow that technique back.

The only downside is that messages on the wire aren't human-readable. If you can live without that and need high performance, consider switching to BooPickle.

uPickle isnt ian apples-to-apples comparison because it doesn't handle shared objects in the pickled graph.

[benhutchison]

I suspect this might be a source of poor scaling, default Seq is a linked list: https://github.com/benhutchison/prickle/blob/master/shared/src/main/scala/prickle/Unpickler.scala#L189

[antonkulaga]

Keep in mind, BooPickle is a son-of-Prickle anyway, clearly derived from the Prickle sources. It innovates

Actually it would be nice if there was one facade with Composite type features and different engines for pickling/unpickling like it is with jawn ( https://github.com/non/jawn ) that supports external AST. In such case developer will choose Prickle with either Boo binary engine or microjson engine with an ability to switch with several lines of code.

[benhutchison]

Well, the PReader and PWriter traits in Prickle are intended to abstract out the write protocol, so in theory they could support a binary encoding.

I wrote the last comment before I'd seen the gitter thread details. JSON vs Binary doesnt account for the massive difference. Turn out there are a bunch of problems in Prickle & microjson which these perf tests have revealed.

Im working on fixing them ATM. Although slower, I hope Prickle will being in the same ballpark as the alternatives afterwards.

On Mon, Apr 20, 2015 at 5:17 AM, Anton Kulaga notifications@github.com wrote:

Keep in mind, BooPickle is a son-of-Prickle anyway, clearly derived from the Prickle sources. It innovates

Actually it would be nice if there was one facade with Composite type features and different engines for pickling/unpickling like it is with jawn ( https://github.com/non/jawn ) that supports external AST. In such case developer will choose Prickle with either Boo binary engine or microjson engine with an ability to switch with several lines of code.

— Reply to this email directly or view it on GitHub https://github.com/benhutchison/prickle/issues/25#issuecomment-94306417.

[benhutchison]

Version 1.1.5 released with measurably improved performance, but still further issues to address Measure using BooPickle JVM benchmarks 1.1.4

1/18 : Encode single Seq[Int]
=============================
Library    ops/s      %          size       %          size.gz    %
BooPickle  193179     7.8%       2          100%       22         100%
Prickle    252226     10.2%      29         1450%      47         214%
uPickle    2463078    100.0%     3          150%       23         105%

2/18 : Decode single Seq[Int]
=============================
Library    ops/s      %          size       %          size.gz    %
BooPickle  3165060    100.0%     2          100%       22         100%
Prickle    32612      1.0%       29         1450%      47         214%
uPickle    1387574    43.8%      3          150%       23         105%

3/18 : Encode very large Seq[Int]
=================================
Library    ops/s      %          size       %          size.gz    %
BooPickle  6404       100.0%     18084      100%       14613      100%
Prickle    602        9.4%       71180      394%       18368      126%
uPickle    762        11.9%      41157      228%       16665      114%

4/18 : Decode very large Seq[Int]
=================================
Library    ops/s      %          size       %          size.gz    %
BooPickle  4346       100.0%     18084      100%       14613      100%
Prickle    3          0.1%       71180      394%       18368      126%
uPickle    524        12.1%      41157      228%       16665      114%

5/18 : Encode large Seq[Double]
===============================
Library    ops/s      %          size       %          size.gz    %
BooPickle  60926      100.0%     8002       100%       7686       100%
Prickle    2108       3.5%       19342      242%       9993       130%
uPickle    2230       3.7%       18319      229%       9780       127%

6/18 : Decode large Seq[Double]
===============================
Library    ops/s      %          size       %          size.gz    %
BooPickle  51270      100.0%     8002       100%       7686       100%
Prickle    261        0.5%       19342      242%       9993       130%
uPickle    2142       4.2%       18319      229%       9780       127%

7/18 : Encode large Seq[Float]
==============================
Library    ops/s      %          size       %          size.gz    %
BooPickle  94748      100.0%     4002       100%       3664       100%
Prickle    1934       2.0%       19031      476%       6872       188%
uPickle    2204       2.3%       18006      450%       6743       184%

8/18 : Decode large Seq[Float]
==============================
Library    ops/s      %          size       %          size.gz    %
BooPickle  63376      100.0%     4002       100%       3664       100%
Prickle    260        0.4%       19031      476%       6872       188%
uPickle    2066       3.3%       18006      450%       6743       184%

9/18 : Encode an object tree
============================
Library    ops/s      %          size       %          size.gz    %
BooPickle  6444       100.0%     3544       100%       1750       100%
Prickle    260        4.0%       33522      946%       4115       235%
uPickle    2366       36.7%      20385      575%       2245       128%

10/18 : Decode an object tree
=============================
Library    ops/s      %          size       %          size.gz    %
BooPickle  17956      100.0%     3544       100%       1750       100%
Prickle    104        0.6%       33522      946%       4115       235%
uPickle    1822       10.1%      20385      575%       2245       128%

11/18 : Encode very large Map[String, Int]
==========================================
Library    ops/s      %          size       %          size.gz    %
BooPickle  345        100.0%     86974      100%       45404      100%
Prickle    1          0.3%       190070     219%       47909      106%
uPickle    327        94.8%      130047     150%       45263      100%

12/18 : Decode very large Map[String, Int]
==========================================
Library    ops/s      %          size       %          size.gz    %
BooPickle  324        100.0%     86974      100%       45404      100%
Prickle    3          0.9%       190070     219%       47909      106%
uPickle    248        76.5%      130047     150%       45263      100%

13/18 : Encoding Seq[Book] with random IDs
==========================================
Library    ops/s      %          size       %          size.gz    %
BooPickle  147678     100.0%     264        100%       241        100%
Prickle    31590      21.4%      933        353%       318        132%
uPickle    110504     74.8%      734        278%       278        115%

14/18 : Decoding Seq[Book] with random IDs
==========================================
Library    ops/s      %          size       %          size.gz    %
BooPickle  410424     100.0%     264        100%       241        100%
Prickle    8008       2.0%       933        353%       318        132%
uPickle    85760      20.9%      734        278%       278        115%

15/18 : Encoding Seq[Book] with UUIDs
=====================================
Library    ops/s      %          size       %          size.gz    %
BooPickle  53768      54.1%      304        100%       315        100%
Prickle    26404      26.6%      1133       373%       454        144%
uPickle    99400      100.0%     934        307%       408        130%

16/18 : Decoding Seq[Book] with UUIDs
=====================================
Library    ops/s      %          size       %          size.gz    %
BooPickle  138542     100.0%     304        100%       315        100%
Prickle    8854       6.4%       1133       373%       454        144%
uPickle    85272      61.5%      934        307%       408        130%

17/18 : Encoding Seq[Book] with numerical IDs
=============================================
Library    ops/s      %          size       %          size.gz    %
BooPickle  142154     100.0%     192        100%       184        100%
Prickle    31460      22.1%      877        457%       269        146%
uPickle    116238     81.8%      678        353%       227        123%

18/18 : Decoding Seq[Book] with numerical IDs
=============================================
Library    ops/s      %          size       %          size.gz    %
BooPickle  528062     100.0%     192        100%       184        100%
Prickle    9342       1.8%       877        457%       269        146%
uPickle    85972      16.3%      678        353%       227        123%

1.1.5:

1/18 : Encode single Seq[Int]
=============================
Library    ops/s      %          size       %          size.gz    %
BooPickle  126094     12.8%      2          100%       22         100%
Prickle    175862     17.8%      29         1450%      47         214%
uPickle    986448     100.0%     3          150%       23         105%

2/18 : Decode single Seq[Int]
=============================
Library    ops/s      %          size       %          size.gz    %
BooPickle  4045488    100.0%     2          100%       22         100%
Prickle    24136      0.6%       29         1450%      47         214%
uPickle    1376320    34.0%      3          150%       23         105%

3/18 : Encode very large Seq[Int]
=================================
Library    ops/s      %          size       %          size.gz    %
BooPickle  6036       100.0%     18084      100%       14613      100%
Prickle    796        13.2%      71180      394%       18368      126%
uPickle    766        12.7%      41157      228%       16665      114%

4/18 : Decode very large Seq[Int]
=================================
Library    ops/s      %          size       %          size.gz    %
BooPickle  4506       100.0%     18084      100%       14613      100%
Prickle    238        5.3%       71180      394%       18368      126%
uPickle    618        13.7%      41157      228%       16665      114%

5/18 : Encode large Seq[Double]
===============================
Library    ops/s      %          size       %          size.gz    %
BooPickle  53008      100.0%     8002       100%       7686       100%
Prickle    2188       4.1%       19342      242%       9993       130%
uPickle    2228       4.2%       18319      229%       9780       127%

6/18 : Decode large Seq[Double]
===============================
Library    ops/s      %          size       %          size.gz    %
BooPickle  57522      100.0%     8002       100%       7686       100%
Prickle    1038       1.8%       19342      242%       9993       130%
uPickle    2410       4.2%       18319      229%       9780       127%

7/18 : Encode large Seq[Float]
==============================
Library    ops/s      %          size       %          size.gz    %
BooPickle  89008      100.0%     4002       100%       3664       100%
Prickle    2282       2.6%       19031      476%       6872       188%
uPickle    2194       2.5%       18006      450%       6743       184%

8/18 : Decode large Seq[Float]
==============================
Library    ops/s      %          size       %          size.gz    %
BooPickle  62158      100.0%     4002       100%       3664       100%
Prickle    1008       1.6%       19031      476%       6872       188%
uPickle    2278       3.7%       18006      450%       6743       184%

9/18 : Encode an object tree
============================
Library    ops/s      %          size       %          size.gz    %
BooPickle  1724       83.0%      3544       100%       1750       100%
Prickle    572        27.6%      35481      1001%      4169       238%
uPickle    2076       100.0%     20385      575%       2245       128%

10/18 : Decode an object tree
=============================
Library    ops/s      %          size       %          size.gz    %
BooPickle  17796      100.0%     3544       100%       1750       100%
Prickle    93         0.5%       35481      1001%      4169       238%
uPickle    2166       12.2%      20385      575%       2245       128%

11/18 : Encode very large Map[String, Int]
==========================================
Library    ops/s      %          size       %          size.gz    %
BooPickle  294        61.3%      86974      100%       45404      100%
Prickle    264        55.0%      190070     219%       47909      106%
uPickle    480        100.0%     130047     150%       45263      100%

12/18 : Decode very large Map[String, Int]
==========================================
Library    ops/s      %          size       %          size.gz    %
BooPickle  335        100.0%     86974      100%       45404      100%
Prickle    66         19.7%      190070     219%       47909      106%
uPickle    247        73.7%      130047     150%       45263      100%

13/18 : Encoding Seq[Book] with random IDs
==========================================
Library    ops/s      %          size       %          size.gz    %
BooPickle  108038     98.7%      264        100%       241        100%
Prickle    32480      29.7%      933        353%       318        132%
uPickle    109476     100.0%     734        278%       278        115%

14/18 : Decoding Seq[Book] with random IDs
==========================================
Library    ops/s      %          size       %          size.gz    %
BooPickle  428118     100.0%     264        100%       241        100%
Prickle    8484       2.0%       933        353%       318        132%
uPickle    101396     23.7%      734        278%       278        115%

15/18 : Encoding Seq[Book] with UUIDs
=====================================
Library    ops/s      %          size       %          size.gz    %
BooPickle  34424      37.9%      304        100%       315        100%
Prickle    28440      31.3%      1133       373%       454        144%
uPickle    90930      100.0%     934        307%       408        130%

16/18 : Decoding Seq[Book] with UUIDs
=====================================
Library    ops/s      %          size       %          size.gz    %
BooPickle  184758     100.0%     304        100%       315        100%
Prickle    8324       4.5%       1133       373%       454        144%
uPickle    94470      51.1%      934        307%       408        130%

17/18 : Encoding Seq[Book] with numerical IDs
=============================================
Library    ops/s      %          size       %          size.gz    %
BooPickle  113160     100.0%     192        100%       184        100%
Prickle    34154      30.2%      877        457%       269        146%
uPickle    111980     99.0%      678        353%       227        123%

18/18 : Decoding Seq[Book] with numerical IDs
=============================================
Library    ops/s      %          size       %          size.gz    %
BooPickle  528324     100.0%     192        100%       184        100%
Prickle    8370       1.6%       877        457%       269        146%
uPickle    101144     19.1%      678        353%       227        123%

[ochrons]

I updated BooPickle perftests to use 1.1.5 Prickle (also the publicly available test site is updated)

[ochrons]

One issue I noticed with Prickle when profiling the performance tests was that there were a lot of exceptions thrown in some of the Prickle tests. You might want to investigate what is causing them.

Also, if a function has a throw statement, it prevents JIT optimization of that function by the JS engine (at least V8).

[benhutchison]

Thanks Otto. Next time Im working in Prickle I'll try to replicate and fix the problem.

I use Prickle with small-medium messages and the lower performance hasn't been noticable, so I havent directed alot of energy towards optimisation.

Whats I'd ideally like is something binary like Boopickle, except with the ability to fallback to json in dev-mode when debugging/tracing is needed.

The other goal I might work on one day is a version of prickle based on typeclasses rather than java equality and hashcode.