It compiles now. How do I run it?

[RuedigerMoeller]

TestRunner is the main. Have look, there are command line options which tests to run. Start with basic stuff excluding tests involving Externalizable. To exclude serializers just temp edit the main. Actually I am in the middle of transition, so a lot of changes are in with-fst2.0 branch. Just tweak the testrunner manually for now.

ias far i remember I run json/cross paltofrm tests with "-cases abcdefghijklmonp"

[RuedigerMoeller]

I merged from branch to master, there is now a pure JSon variant.

run with TestRunner -var cross

results from my win machine are here: https://github.com/RuedigerMoeller/serialization-benchmark/wiki/json-only

There are many failures. The original test was designed for full object serializers. If you think some of the testcases are too complex or contain stuff which cannot be done in JSon, let me now. I would then add new tests cases designed for JSon/Cross Platform encoding.

JSON-IO has the fewest failures but is slowest. Jackson seems very fast to me with databinding ..

[RichardHightower]

I introduced a feature around .13 or .11 that is slowing down serialization a lot. There is an expensive call in seeializeInstance.

It can be turned off with a flag in .16 or .17.

This feature could also really help with serialization speed and size.

It allows for sideways serialization. Serializing objects to lists instead of maps.

I noticed the size delta between Jackson and Boon JSON. Boon only delivers compliant JSON. And it delivers the smallest possible complaint JSON. To get smaller you would have to deliver non compliant JSON. Boon does not do that. It does deliver sideways JSON but I have never benchmarked the output serialization of it.

On Saturday, May 10, 2014, RuedigerMoeller notifications@github.com wrote:

I merged from branch to master, there is now a pure JSon variant.

run with TestRunner -var cross

results from my win machine are here: https://github.com/RuedigerMoeller/serialization-benchmark/wiki/json-only

There are many failures. The original test was designed for full object serializers. If you think some of the testcases are too complex or contain stuff which cannot be done in JSon, let me now. I would then add new tests cases designed for JSon/Cross Platform encoding.

JSON-IO has the fewest failures but is slowest. Jackson seems very fast to me with databinding ..

— Reply to this email directly or view it on GitHubhttps://github.com/RuedigerMoeller/serialization-benchmark/issues/2#issuecomment-42741507 .

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

[RichardHightower]

Try this.... setSerializeAsSupport(false)

That should speed things up for Boon by about 20% to 25%.

private final JsonSerializer serializer = new JsonSerializerFactory()

.setSerializeAsSupport(false).setEncodeStrings(false).useFieldsOnly().create();

private final JsonSerializer serializer = new JsonSerializerFactory()
        .setSerializeAsSupport(false).useFieldsOnly().create();

private final JsonParserAndMapper parser = new

JsonParserFactory().create();

But if Jackson is outputting and reading non-standard JSON then there is only so close Boon can get. Boon can read non-standard JSON but it can't output it.

BTW Jackson does a really good job on this test. Congrats to Tatu. I don't mind losing.

It would be good to see the delta if Jackson was forced to output standard JSON.

But Tatu and Jackson are kicking but at this test. :)

On Sat, May 10, 2014 at 10:11 AM, Rick Hightower <richardhightower@gmail.com

wrote:

I introduced a feature around .13 or .11 that is slowing down serialization a lot. There is an expensive call in seeializeInstance.

It can be turned off with a flag in .16 or .17.

This feature could also really help with serialization speed and size.

It allows for sideways serialization. Serializing objects to lists instead of maps.

I noticed the size delta between Jackson and Boon JSON. Boon only delivers compliant JSON. And it delivers the smallest possible complaint JSON. To get smaller you would have to deliver non compliant JSON. Boon does not do that. It does deliver sideways JSON but I have never benchmarked the output serialization of it.

On Saturday, May 10, 2014, RuedigerMoeller notifications@github.com wrote:

I merged from branch to master, there is now a pure JSon variant.

run with TestRunner -var cross

results from my win machine are here: https://github.com/RuedigerMoeller/serialization-benchmark/wiki/json-only

There are many failures. The original test was designed for full object serializers. If you think some of the testcases are too complex or contain stuff which cannot be done in JSon, let me now. I would then add new tests cases designed for JSon/Cross Platform encoding.

JSON-IO has the fewest failures but is slowest. Jackson seems very fast to me with databinding ..

— Reply to this email directly or view it on GitHubhttps://github.com/RuedigerMoeller/serialization-benchmark/issues/2#issuecomment-42741507 .

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

[RichardHightower]

I don't understand the results of:

ManyClasses What does 0 ns mean? Looks like a tie. Trader

What does 0 ns mean? MediaContent I added to my test suite, and Jackson and Boon are much closer with JMH. ShortRemoteCall What does 0 ns mean? So did Boon win this one? Test results seem odd when the results are 0 ns.

On Sat, May 10, 2014 at 10:25 AM, Rick Hightower <richardhightower@gmail.com

wrote:

Try this.... setSerializeAsSupport(false)

That should speed things up for Boon by about 20% to 25%.

private final JsonSerializer serializer = new JsonSerializerFactory()

.setSerializeAsSupport(false).setEncodeStrings(false).useFieldsOnly().create();

private final JsonSerializer serializer = new JsonSerializerFactory()
        .setSerializeAsSupport(false).useFieldsOnly().create();

private final JsonParserAndMapper parser = new

JsonParserFactory().create();

But if Jackson is outputting and reading non-standard JSON then there is only so close Boon can get. Boon can read non-standard JSON but it can't output it.

BTW Jackson does a really good job on this test. Congrats to Tatu. I don't mind losing.

It would be good to see the delta if Jackson was forced to output standard JSON.

But Tatu and Jackson are kicking but at this test. :)

On Sat, May 10, 2014 at 10:11 AM, Rick Hightower < richardhightower@gmail.com> wrote:

I introduced a feature around .13 or .11 that is slowing down serialization a lot. There is an expensive call in seeializeInstance.

It can be turned off with a flag in .16 or .17.

This feature could also really help with serialization speed and size.

It allows for sideways serialization. Serializing objects to lists instead of maps.

I noticed the size delta between Jackson and Boon JSON. Boon only delivers compliant JSON. And it delivers the smallest possible complaint JSON. To get smaller you would have to deliver non compliant JSON. Boon does not do that. It does deliver sideways JSON but I have never benchmarked the output serialization of it.

On Saturday, May 10, 2014, RuedigerMoeller notifications@github.com wrote:

I merged from branch to master, there is now a pure JSon variant.

run with TestRunner -var cross

results from my win machine are here: https://github.com/RuedigerMoeller/serialization-benchmark/wiki/json-only

There are many failures. The original test was designed for full object serializers. If you think some of the testcases are too complex or contain stuff which cannot be done in JSon, let me now. I would then add new tests cases designed for JSon/Cross Platform encoding.

JSON-IO has the fewest failures but is slowest. Jackson seems very fast to me with databinding ..

— Reply to this email directly or view it on GitHubhttps://github.com/RuedigerMoeller/serialization-benchmark/issues/2#issuecomment-42741507 .

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

[RuedigerMoeller]

0 means failure either because of an exception or because deepequals fails.

Am Samstag, 10. Mai 2014 schrieb Richard Hightower :

I don't understand the results of:

ManyClasses What does 0 ns mean? Looks like a tie. Trader

What does 0 ns mean? MediaContent I added to my test suite, and Jackson and Boon are much closer with JMH. ShortRemoteCall What does 0 ns mean? So did Boon win this one? Test results seem odd when the results are 0 ns.

On Sat, May 10, 2014 at 10:25 AM, Rick Hightower < richardhightower@gmail.comjavascript:_e(%7B%7D,'cvml','richardhightower@gmail.com');

wrote:

Try this.... setSerializeAsSupport(false)

That should speed things up for Boon by about 20% to 25%.

private final JsonSerializer serializer = new JsonSerializerFactory()

.setSerializeAsSupport(false).setEncodeStrings(false).useFieldsOnly().create();

private final JsonSerializer serializer = new JsonSerializerFactory() .setSerializeAsSupport(false).useFieldsOnly().create();

private final JsonParserAndMapper parser = new JsonParserFactory().create();

But if Jackson is outputting and reading non-standard JSON then there is only so close Boon can get. Boon can read non-standard JSON but it can't output it.

BTW Jackson does a really good job on this test. Congrats to Tatu. I don't mind losing.

It would be good to see the delta if Jackson was forced to output standard JSON.

But Tatu and Jackson are kicking but at this test. :)

On Sat, May 10, 2014 at 10:11 AM, Rick Hightower < richardhightower@gmail.comjavascript:_e(%7B%7D,'cvml','richardhightower@gmail.com');> wrote:

I introduced a feature around .13 or .11 that is slowing down serialization a lot. There is an expensive call in seeializeInstance.

It can be turned off with a flag in .16 or .17.

This feature could also really help with serialization speed and size.

It allows for sideways serialization. Serializing objects to lists instead of maps.

I noticed the size delta between Jackson and Boon JSON. Boon only delivers compliant JSON. And it delivers the smallest possible complaint JSON. To get smaller you would have to deliver non compliant JSON. Boon does not do that. It does deliver sideways JSON but I have never benchmarked the output serialization of it.

On Saturday, May 10, 2014, RuedigerMoeller notifications@github.com<javascript:_e(%7B%7D,'cvml','notifications@github.com');>

wrote:

I merged from branch to master, there is now a pure JSon variant.

run with TestRunner -var cross

results from my win machine are here:

https://github.com/RuedigerMoeller/serialization-benchmark/wiki/json-only

There are many failures. The original test was designed for full object serializers. If you think some of the testcases are too complex or contain stuff which cannot be done in JSon, let me now. I would then add new tests cases designed for JSon/Cross Platform encoding.

JSON-IO has the fewest failures but is slowest. Jackson seems very fast to me with databinding ..

— Reply to this email directly or view it on GitHub< https://github.com/RuedigerMoeller/serialization-benchmark/issues/2#issuecomment-42741507>

.

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

— Reply to this email directly or view it on GitHubhttps://github.com/RuedigerMoeller/serialization-benchmark/issues/2#issuecomment-42748591 .

[RichardHightower]

i.g.j.s.MainJacksonSerializer.mediaContentRoundTrip     thrpt  16
  3    1    43426.828     1496.530    ops/s

.g.j.s.MainBoonSerializer.mediaContentRoundTrip        thrpt  16         3
  1    44146.583    27928.308    ops/s

When I test media content round trip with JMH, Jackson and Boon are tied.

Jackson is faster at output. Boon is faster at parse. And they meet in the middle.

i.g.j.s.MainJacksonSerializer.roundTripBig              thrpt  16
3    1       77.306       i.g.j.s.MainBoonSerializer.roundTripBig
      thrpt  16         3    1       88.017

i.g.j.s.MainBoonSerializer.roundTriper                  thrpt  16
3    1   376859.083

i.g.j.s.MainJacksonSerializer.roundTriper               thrpt  16
3    1   222502.956

Other serialization tests Boon does pretty well. But... I have never tested serializing a big random float array so. :) I can concede victory, and tune boon.

I will add your tests to the Boon JMH benchmark and see if there is anything I can do to make Boon faster for these test cases. I will also try to run your test suite again. It would be nice if that was on the wiki. I assume you just need to cut and paste how you run the tests from you command prompt to the wiki page. :)

I run my serialization tests like this:

java -jar target/microbenchmarks.jar "._serialization._Main.*" -wi 2 -i 3 -f 0 -t 16

I plan on moving your tests one by one over to JMH. :)

The 0 ns score ruffles my feathers.

On Sat, May 10, 2014 at 10:32 AM, Rick Hightower <richardhightower@gmail.com

wrote:

I don't understand the results of:

ManyClasses What does 0 ns mean? Looks like a tie. Trader

What does 0 ns mean? MediaContent I added to my test suite, and Jackson and Boon are much closer with JMH. ShortRemoteCall What does 0 ns mean? So did Boon win this one? Test results seem odd when the results are 0 ns.

On Sat, May 10, 2014 at 10:25 AM, Rick Hightower < richardhightower@gmail.com> wrote:

Try this.... setSerializeAsSupport(false)

That should speed things up for Boon by about 20% to 25%.

private final JsonSerializer serializer = new JsonSerializerFactory()

.setSerializeAsSupport(false).setEncodeStrings(false).useFieldsOnly().create();

private final JsonSerializer serializer = new JsonSerializerFactory()
        .setSerializeAsSupport(false).useFieldsOnly().create();

private final JsonParserAndMapper parser = new

JsonParserFactory().create();

But if Jackson is outputting and reading non-standard JSON then there is only so close Boon can get. Boon can read non-standard JSON but it can't output it.

BTW Jackson does a really good job on this test. Congrats to Tatu. I don't mind losing.

It would be good to see the delta if Jackson was forced to output standard JSON.

But Tatu and Jackson are kicking but at this test. :)

On Sat, May 10, 2014 at 10:11 AM, Rick Hightower < richardhightower@gmail.com> wrote:

I introduced a feature around .13 or .11 that is slowing down serialization a lot. There is an expensive call in seeializeInstance.

It can be turned off with a flag in .16 or .17.

This feature could also really help with serialization speed and size.

It allows for sideways serialization. Serializing objects to lists instead of maps.

I noticed the size delta between Jackson and Boon JSON. Boon only delivers compliant JSON. And it delivers the smallest possible complaint JSON. To get smaller you would have to deliver non compliant JSON. Boon does not do that. It does deliver sideways JSON but I have never benchmarked the output serialization of it.

On Saturday, May 10, 2014, RuedigerMoeller notifications@github.com wrote:

I merged from branch to master, there is now a pure JSon variant.

run with TestRunner -var cross

results from my win machine are here:

https://github.com/RuedigerMoeller/serialization-benchmark/wiki/json-only

There are many failures. The original test was designed for full object serializers. If you think some of the testcases are too complex or contain stuff which cannot be done in JSon, let me now. I would then add new tests cases designed for JSon/Cross Platform encoding.

JSON-IO has the fewest failures but is slowest. Jackson seems very fast to me with databinding ..

— Reply to this email directly or view it on GitHubhttps://github.com/RuedigerMoeller/serialization-benchmark/issues/2#issuecomment-42741507 .

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

[RichardHightower]

Ah... fair enough. I have a unit test for MediaContent in boon proper that checks deep equals but did not add this to the benchmark.

On Sat, May 10, 2014 at 10:42 AM, RuedigerMoeller notifications@github.comwrote:

0 means failure either because of an exception or because deepequals fails.

Am Samstag, 10. Mai 2014 schrieb Richard Hightower :

I don't understand the results of:

ManyClasses What does 0 ns mean? Looks like a tie. Trader

What does 0 ns mean? MediaContent I added to my test suite, and Jackson and Boon are much closer with JMH. ShortRemoteCall What does 0 ns mean? So did Boon win this one? Test results seem odd when the results are 0 ns.

On Sat, May 10, 2014 at 10:25 AM, Rick Hightower < richardhightower@gmail.com<javascript:_e(%7B%7D,'cvml',' richardhightower@gmail.com');>

wrote:

Try this.... setSerializeAsSupport(false)

That should speed things up for Boon by about 20% to 25%.

private final JsonSerializer serializer = new JsonSerializerFactory()

.setSerializeAsSupport(false).setEncodeStrings(false).useFieldsOnly().create();

private final JsonSerializer serializer = new JsonSerializerFactory() .setSerializeAsSupport(false).useFieldsOnly().create();

private final JsonParserAndMapper parser = new JsonParserFactory().create();

But if Jackson is outputting and reading non-standard JSON then there is only so close Boon can get. Boon can read non-standard JSON but it can't output it.

BTW Jackson does a really good job on this test. Congrats to Tatu. I don't mind losing.

It would be good to see the delta if Jackson was forced to output standard JSON.

But Tatu and Jackson are kicking but at this test. :)

On Sat, May 10, 2014 at 10:11 AM, Rick Hightower < richardhightower@gmail.com<javascript:_e(%7B%7D,'cvml',' richardhightower@gmail.com');>> wrote:

I introduced a feature around .13 or .11 that is slowing down serialization a lot. There is an expensive call in seeializeInstance.

It can be turned off with a flag in .16 or .17.

This feature could also really help with serialization speed and size.

It allows for sideways serialization. Serializing objects to lists instead of maps.

I noticed the size delta between Jackson and Boon JSON. Boon only delivers compliant JSON. And it delivers the smallest possible complaint JSON. To get smaller you would have to deliver non compliant JSON. Boon does not do that. It does deliver sideways JSON but I have never benchmarked the output serialization of it.

On Saturday, May 10, 2014, RuedigerMoeller <notifications@github.com javascript:_e(%7B%7D,'cvml','notifications@github.com');>

wrote:

I merged from branch to master, there is now a pure JSon variant.

run with TestRunner -var cross

results from my win machine are here:

https://github.com/RuedigerMoeller/serialization-benchmark/wiki/json-only

There are many failures. The original test was designed for full object serializers. If you think some of the testcases are too complex or contain stuff which cannot be done in JSon, let me now. I would then add new tests cases designed for JSon/Cross Platform encoding.

JSON-IO has the fewest failures but is slowest. Jackson seems very fast to me with databinding ..

— Reply to this email directly or view it on GitHub<

https://github.com/RuedigerMoeller/serialization-benchmark/issues/2#issuecomment-42741507>

.

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

— Reply to this email directly or view it on GitHub< https://github.com/RuedigerMoeller/serialization-benchmark/issues/2#issuecomment-42748591>

.

— Reply to this email directly or view it on GitHubhttps://github.com/RuedigerMoeller/serialization-benchmark/issues/2#issuecomment-42748886 .

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

[RichardHightower]

Benchmark                                    Mode Thr     Count  Sec
  Mean   Mean error    Units

i.g.j.s.MainBoonSerializer.stringPerf       thrpt   5         3    1
8380.844     3565.429    ops/s

i.g.j.s.MainJacksonSerializer.stringPerf    thrpt   5         3    1
23136.272     6862.268    ops/s

I have reproduced what you see with String performance. I don't think I handle arrays very well. I typically use List and Maps. I don't see that many arrays being used for REST / Websocket, but... but.. .but...

You know me. Let met try a few things.

On Sat, May 10, 2014 at 10:50 AM, Rick Hightower <richardhightower@gmail.com

wrote:

Ah... fair enough. I have a unit test for MediaContent in boon proper that checks deep equals but did not add this to the benchmark.

On Sat, May 10, 2014 at 10:42 AM, RuedigerMoeller < notifications@github.com> wrote:

0 means failure either because of an exception or because deepequals fails.

Am Samstag, 10. Mai 2014 schrieb Richard Hightower :

I don't understand the results of:

ManyClasses What does 0 ns mean? Looks like a tie. Trader

What does 0 ns mean? MediaContent I added to my test suite, and Jackson and Boon are much closer with JMH. ShortRemoteCall What does 0 ns mean? So did Boon win this one? Test results seem odd when the results are 0 ns.

On Sat, May 10, 2014 at 10:25 AM, Rick Hightower < richardhightower@gmail.com<javascript:_e(%7B%7D,'cvml',' richardhightower@gmail.com');>

wrote:

Try this.... setSerializeAsSupport(false)

That should speed things up for Boon by about 20% to 25%.

private final JsonSerializer serializer = new JsonSerializerFactory()

.setSerializeAsSupport(false).setEncodeStrings(false).useFieldsOnly().create();

private final JsonSerializer serializer = new JsonSerializerFactory() .setSerializeAsSupport(false).useFieldsOnly().create();

private final JsonParserAndMapper parser = new JsonParserFactory().create();

But if Jackson is outputting and reading non-standard JSON then there is only so close Boon can get. Boon can read non-standard JSON but it can't output it.

BTW Jackson does a really good job on this test. Congrats to Tatu. I don't mind losing.

It would be good to see the delta if Jackson was forced to output standard JSON.

But Tatu and Jackson are kicking but at this test. :)

On Sat, May 10, 2014 at 10:11 AM, Rick Hightower < richardhightower@gmail.com<javascript:_e(%7B%7D,'cvml',' richardhightower@gmail.com');>> wrote:

I introduced a feature around .13 or .11 that is slowing down serialization a lot. There is an expensive call in seeializeInstance.

It can be turned off with a flag in .16 or .17.

This feature could also really help with serialization speed and size.

It allows for sideways serialization. Serializing objects to lists instead of maps.

I noticed the size delta between Jackson and Boon JSON. Boon only delivers compliant JSON. And it delivers the smallest possible complaint JSON. To get smaller you would have to deliver non compliant JSON. Boon does not do that. It does deliver sideways JSON but I have never benchmarked the output serialization of it.

On Saturday, May 10, 2014, RuedigerMoeller <notifications@github.com javascript:_e(%7B%7D,'cvml','notifications@github.com');>

wrote:

I merged from branch to master, there is now a pure JSon variant.

run with TestRunner -var cross

results from my win machine are here:

https://github.com/RuedigerMoeller/serialization-benchmark/wiki/json-only

There are many failures. The original test was designed for full object serializers. If you think some of the testcases are too complex or contain stuff which cannot be done in JSon, let me now. I would then add new tests cases designed for JSon/Cross Platform encoding.

JSON-IO has the fewest failures but is slowest. Jackson seems very fast to me with databinding ..

— Reply to this email directly or view it on GitHub<

https://github.com/RuedigerMoeller/serialization-benchmark/issues/2#issuecomment-42741507>

.

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

— Reply to this email directly or view it on GitHub< https://github.com/RuedigerMoeller/serialization-benchmark/issues/2#issuecomment-42748591>

.

— Reply to this email directly or view it on GitHubhttps://github.com/RuedigerMoeller/serialization-benchmark/issues/2#issuecomment-42748886 .

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

[RichardHightower]

I switched to some tighter array handing and that did not matter.

He was two x faster at encoding strings, and now he is three x (I ran against older Jackson instances).

So it looks like Jackson is just faster at encoding strings period. (was 2x and now 3x)

I will break the benchmark down into two.

One for parse and one for encoding (output).

Thanks.

I did not know how much faster Jackson was at handling Strings.

This should help Boon with all of the benchmarks. :)

I am sure I can improve encoding and decoding.

On Sat, May 10, 2014 at 4:41 PM, Rick Hightower richardhightower@gmail.comwrote:

Benchmark                                    Mode Thr     Count  Sec
  Mean   Mean error    Units

i.g.j.s.MainBoonSerializer.stringPerf       thrpt   5         3    1
8380.844     3565.429    ops/s

i.g.j.s.MainJacksonSerializer.stringPerf    thrpt   5         3    1
23136.272     6862.268    ops/s

I have reproduced what you see with String performance. I don't think I handle arrays very well. I typically use List and Maps. I don't see that many arrays being used for REST / Websocket, but... but.. .but...

You know me. Let met try a few things.

On Sat, May 10, 2014 at 10:50 AM, Rick Hightower < richardhightower@gmail.com> wrote:

Ah... fair enough. I have a unit test for MediaContent in boon proper that checks deep equals but did not add this to the benchmark.

On Sat, May 10, 2014 at 10:42 AM, RuedigerMoeller < notifications@github.com> wrote:

0 means failure either because of an exception or because deepequals fails.

Am Samstag, 10. Mai 2014 schrieb Richard Hightower :

I don't understand the results of:

ManyClasses What does 0 ns mean? Looks like a tie. Trader

What does 0 ns mean? MediaContent I added to my test suite, and Jackson and Boon are much closer with JMH. ShortRemoteCall What does 0 ns mean? So did Boon win this one? Test results seem odd when the results are 0 ns.

On Sat, May 10, 2014 at 10:25 AM, Rick Hightower < richardhightower@gmail.com<javascript:_e(%7B%7D,'cvml',' richardhightower@gmail.com');>

wrote:

Try this.... setSerializeAsSupport(false)

That should speed things up for Boon by about 20% to 25%.

private final JsonSerializer serializer = new JsonSerializerFactory()

.setSerializeAsSupport(false).setEncodeStrings(false).useFieldsOnly().create();

private final JsonSerializer serializer = new JsonSerializerFactory() .setSerializeAsSupport(false).useFieldsOnly().create();

private final JsonParserAndMapper parser = new JsonParserFactory().create();

But if Jackson is outputting and reading non-standard JSON then there is only so close Boon can get. Boon can read non-standard JSON but it can't output it.

BTW Jackson does a really good job on this test. Congrats to Tatu. I don't mind losing.

It would be good to see the delta if Jackson was forced to output standard JSON.

But Tatu and Jackson are kicking but at this test. :)

On Sat, May 10, 2014 at 10:11 AM, Rick Hightower < richardhightower@gmail.com<javascript:_e(%7B%7D,'cvml',' richardhightower@gmail.com');>> wrote:

I introduced a feature around .13 or .11 that is slowing down serialization a lot. There is an expensive call in seeializeInstance.

It can be turned off with a flag in .16 or .17.

This feature could also really help with serialization speed and size.

It allows for sideways serialization. Serializing objects to lists instead of maps.

I noticed the size delta between Jackson and Boon JSON. Boon only delivers compliant JSON. And it delivers the smallest possible complaint JSON. To get smaller you would have to deliver non compliant JSON. Boon does not do that. It does deliver sideways JSON but I have never benchmarked the output serialization of it.

On Saturday, May 10, 2014, RuedigerMoeller < notifications@github.com<javascript:_e(%7B%7D,'cvml',' notifications@github.com');>>

wrote:

I merged from branch to master, there is now a pure JSon variant.

run with TestRunner -var cross

results from my win machine are here:

https://github.com/RuedigerMoeller/serialization-benchmark/wiki/json-only

There are many failures. The original test was designed for full object serializers. If you think some of the testcases are too complex or contain stuff which cannot be done in JSon, let me now. I would then add new tests cases designed for JSon/Cross Platform encoding.

JSON-IO has the fewest failures but is slowest. Jackson seems very fast to me with databinding ..

— Reply to this email directly or view it on GitHub<

https://github.com/RuedigerMoeller/serialization-benchmark/issues/2#issuecomment-42741507>

.

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

— Reply to this email directly or view it on GitHub< https://github.com/RuedigerMoeller/serialization-benchmark/issues/2#issuecomment-42748591>

.

— Reply to this email directly or view it on GitHubhttps://github.com/RuedigerMoeller/serialization-benchmark/issues/2#issuecomment-42748886 .

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

[RichardHightower]

Imagine how fast Boon will be when it has a faster encoder.

On Sat, May 10, 2014 at 5:16 PM, Rick Hightower richardhightower@gmail.comwrote:

I switched to some tighter array handing and that did not matter.

He was two x faster at encoding strings, and now he is three x (I ran against older Jackson instances).

So it looks like Jackson is just faster at encoding strings period. (was 2x and now 3x)

I will break the benchmark down into two.

One for parse and one for encoding (output).

Thanks.

I did not know how much faster Jackson was at handling Strings.

This should help Boon with all of the benchmarks. :)

I am sure I can improve encoding and decoding.

On Sat, May 10, 2014 at 4:41 PM, Rick Hightower < richardhightower@gmail.com> wrote:

Benchmark                                    Mode Thr     Count  Sec
    Mean   Mean error    Units

i.g.j.s.MainBoonSerializer.stringPerf       thrpt   5         3    1
8380.844     3565.429    ops/s

i.g.j.s.MainJacksonSerializer.stringPerf    thrpt   5         3    1
23136.272     6862.268    ops/s

I have reproduced what you see with String performance. I don't think I handle arrays very well. I typically use List and Maps. I don't see that many arrays being used for REST / Websocket, but... but.. .but...

You know me. Let met try a few things.

On Sat, May 10, 2014 at 10:50 AM, Rick Hightower < richardhightower@gmail.com> wrote:

Ah... fair enough. I have a unit test for MediaContent in boon proper that checks deep equals but did not add this to the benchmark.

On Sat, May 10, 2014 at 10:42 AM, RuedigerMoeller < notifications@github.com> wrote:

0 means failure either because of an exception or because deepequals fails.

Am Samstag, 10. Mai 2014 schrieb Richard Hightower :

I don't understand the results of:

ManyClasses What does 0 ns mean? Looks like a tie. Trader

What does 0 ns mean? MediaContent I added to my test suite, and Jackson and Boon are much closer with JMH. ShortRemoteCall What does 0 ns mean? So did Boon win this one? Test results seem odd when the results are 0 ns.

On Sat, May 10, 2014 at 10:25 AM, Rick Hightower < richardhightower@gmail.com<javascript:_e(%7B%7D,'cvml',' richardhightower@gmail.com');>

wrote:

Try this.... setSerializeAsSupport(false)

That should speed things up for Boon by about 20% to 25%.

private final JsonSerializer serializer = new JsonSerializerFactory()

.setSerializeAsSupport(false).setEncodeStrings(false).useFieldsOnly().create();

private final JsonSerializer serializer = new JsonSerializerFactory() .setSerializeAsSupport(false).useFieldsOnly().create();

private final JsonParserAndMapper parser = new JsonParserFactory().create();

But if Jackson is outputting and reading non-standard JSON then there is only so close Boon can get. Boon can read non-standard JSON but it can't output it.

BTW Jackson does a really good job on this test. Congrats to Tatu. I don't mind losing.

It would be good to see the delta if Jackson was forced to output standard JSON.

But Tatu and Jackson are kicking but at this test. :)

On Sat, May 10, 2014 at 10:11 AM, Rick Hightower < richardhightower@gmail.com<javascript:_e(%7B%7D,'cvml',' richardhightower@gmail.com');>> wrote:

I introduced a feature around .13 or .11 that is slowing down serialization a lot. There is an expensive call in seeializeInstance.

It can be turned off with a flag in .16 or .17.

This feature could also really help with serialization speed and size.

It allows for sideways serialization. Serializing objects to lists instead of maps.

I noticed the size delta between Jackson and Boon JSON. Boon only delivers compliant JSON. And it delivers the smallest possible complaint JSON. To get smaller you would have to deliver non compliant JSON. Boon does not do that. It does deliver sideways JSON but I have never benchmarked the output serialization of it.

On Saturday, May 10, 2014, RuedigerMoeller < notifications@github.com<javascript:_e(%7B%7D,'cvml',' notifications@github.com');>>

wrote:

I merged from branch to master, there is now a pure JSon variant.

run with TestRunner -var cross

results from my win machine are here:

https://github.com/RuedigerMoeller/serialization-benchmark/wiki/json-only

There are many failures. The original test was designed for full object serializers. If you think some of the testcases are too complex or contain stuff which cannot be done in JSon, let me now. I would then add new tests cases designed for JSon/Cross Platform encoding.

JSON-IO has the fewest failures but is slowest. Jackson seems very fast to me with databinding ..

— Reply to this email directly or view it on GitHub<

https://github.com/RuedigerMoeller/serialization-benchmark/issues/2#issuecomment-42741507>

.

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

— Reply to this email directly or view it on GitHub< https://github.com/RuedigerMoeller/serialization-benchmark/issues/2#issuecomment-42748591>

.

— Reply to this email directly or view it on GitHubhttps://github.com/RuedigerMoeller/serialization-benchmark/issues/2#issuecomment-42748886 .

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

[RichardHightower]

I've been optimizing string perf.

A new version is coming, and it is going to be very, very, very fast.

Looks like I introduced some slow down while fixing a bug for a Groovy regression case.

Before changes...

Benchmark                                              Mode Thr     Count
Sec         Mean   Mean error    Units

i.g.j.s.MainBoonSerializer.stringPerf                 thrpt   5         3
  1     5169.078      170.368    ops/s

i.g.j.s.MainBoonSerializer.stringPerfParser           thrpt   5         3
  1    14020.722     5614.226    ops/s

i.g.j.s.MainBoonSerializer.stringPerfSerializer       thrpt   5         3
  1    14346.744     9297.693    ops/s

i.g.j.s.MainJacksonSerializer.stringPerf              thrpt   5         3
  1    17947.417     2059.609    ops/s

i.g.j.s.MainJacksonSerializer.stringPerfParser        thrpt   5         3
  1    15573.783     3727.068    ops/s

i.g.j.s.MainJacksonSerializer.stringPerfSerializer    thrpt   5         3
  1    46201.461   192468.331    ops/s

Jackson kicking my ass.

Run 2

Benchmark                                              Mode Thr     Count
Sec         Mean   Mean error    Units

i.g.j.s.MainBoonSerializer.stringPerf                 thrpt   5         3
  1     5302.344     1010.807    ops/s

i.g.j.s.MainBoonSerializer.stringPerfParser           thrpt   5         3
  1    13420.678     3676.203    ops/s

i.g.j.s.MainBoonSerializer.stringPerfSerializer       thrpt   5         3
  1    17497.033     1695.500    ops/s

i.g.j.s.MainJacksonSerializer.stringPerf              thrpt   5         3
  1    18296.372     3316.231    ops/s

i.g.j.s.MainJacksonSerializer.stringPerfParser        thrpt   5         3
  1    15346.422      925.723    ops/s

i.g.j.s.MainJacksonSerializer.stringPerfSerializer    thrpt   5         3
  1    64536.778    14897.001    ops/s

Run 3

Benchmark                                              Mode Thr     Count
Sec         Mean   Mean error    Units

i.g.j.s.MainBoonSerializer.stringPerf                 thrpt   5         3
  1    12350.306     1195.551    ops/s

i.g.j.s.MainBoonSerializer.stringPerfParser           thrpt   5         3
  1   121352.889    19517.317    ops/s

i.g.j.s.MainBoonSerializer.stringPerfSerializer       thrpt   5         3
  1   755461.644   473423.086    ops/s

i.g.j.s.MainJacksonSerializer.stringPerf              thrpt   5         3
  1    20124.978     4087.856    ops/s

i.g.j.s.MainJacksonSerializer.stringPerfParser        thrpt   5         3
  1    17125.267     4727.714    ops/s

i.g.j.s.MainJacksonSerializer.stringPerfSerializer    thrpt   5         3
  1    65055.011    12442.403    ops/s

I have a mapping slow down to fix.

But you can see the improvements in the others.

Its coming.

On Sat, May 10, 2014 at 5:16 PM, Rick Hightower richardhightower@gmail.comwrote:

Imagine how fast Boon will be when it has a faster encoder.

On Sat, May 10, 2014 at 5:16 PM, Rick Hightower < richardhightower@gmail.com> wrote:

I switched to some tighter array handing and that did not matter.

He was two x faster at encoding strings, and now he is three x (I ran against older Jackson instances).

So it looks like Jackson is just faster at encoding strings period. (was 2x and now 3x)

I will break the benchmark down into two.

One for parse and one for encoding (output).

Thanks.

I did not know how much faster Jackson was at handling Strings.

This should help Boon with all of the benchmarks. :)

I am sure I can improve encoding and decoding.

On Sat, May 10, 2014 at 4:41 PM, Rick Hightower < richardhightower@gmail.com> wrote:

Benchmark                                    Mode Thr     Count  Sec
    Mean   Mean error    Units

i.g.j.s.MainBoonSerializer.stringPerf       thrpt   5         3    1
8380.844     3565.429    ops/s

i.g.j.s.MainJacksonSerializer.stringPerf    thrpt   5         3    1
23136.272     6862.268    ops/s

I have reproduced what you see with String performance. I don't think I handle arrays very well. I typically use List and Maps. I don't see that many arrays being used for REST / Websocket, but... but.. .but...

You know me. Let met try a few things.

On Sat, May 10, 2014 at 10:50 AM, Rick Hightower < richardhightower@gmail.com> wrote:

Ah... fair enough. I have a unit test for MediaContent in boon proper that checks deep equals but did not add this to the benchmark.

On Sat, May 10, 2014 at 10:42 AM, RuedigerMoeller < notifications@github.com> wrote:

0 means failure either because of an exception or because deepequals fails.

Am Samstag, 10. Mai 2014 schrieb Richard Hightower :

I don't understand the results of:

ManyClasses What does 0 ns mean? Looks like a tie. Trader

What does 0 ns mean? MediaContent I added to my test suite, and Jackson and Boon are much closer with JMH. ShortRemoteCall What does 0 ns mean? So did Boon win this one? Test results seem odd when the results are 0 ns.

On Sat, May 10, 2014 at 10:25 AM, Rick Hightower < richardhightower@gmail.com<javascript:_e(%7B%7D,'cvml',' richardhightower@gmail.com');>

wrote:

Try this.... setSerializeAsSupport(false)

That should speed things up for Boon by about 20% to 25%.

private final JsonSerializer serializer = new JsonSerializerFactory()

.setSerializeAsSupport(false).setEncodeStrings(false).useFieldsOnly().create();

private final JsonSerializer serializer = new JsonSerializerFactory() .setSerializeAsSupport(false).useFieldsOnly().create();

private final JsonParserAndMapper parser = new JsonParserFactory().create();

But if Jackson is outputting and reading non-standard JSON then there is only so close Boon can get. Boon can read non-standard JSON but it can't output it.

BTW Jackson does a really good job on this test. Congrats to Tatu. I don't mind losing.

It would be good to see the delta if Jackson was forced to output standard JSON.

But Tatu and Jackson are kicking but at this test. :)

On Sat, May 10, 2014 at 10:11 AM, Rick Hightower < richardhightower@gmail.com<javascript:_e(%7B%7D,'cvml',' richardhightower@gmail.com');>> wrote:

I introduced a feature around .13 or .11 that is slowing down serialization a lot. There is an expensive call in seeializeInstance.

It can be turned off with a flag in .16 or .17.

This feature could also really help with serialization speed and size.

It allows for sideways serialization. Serializing objects to lists instead of maps.

I noticed the size delta between Jackson and Boon JSON. Boon only delivers compliant JSON. And it delivers the smallest possible complaint JSON. To get smaller you would have to deliver non compliant JSON. Boon does not do that. It does deliver sideways JSON but I have never benchmarked the output serialization of it.

On Saturday, May 10, 2014, RuedigerMoeller < notifications@github.com<javascript:_e(%7B%7D,'cvml',' notifications@github.com');>>

wrote:

I merged from branch to master, there is now a pure JSon variant.

run with TestRunner -var cross

results from my win machine are here:

https://github.com/RuedigerMoeller/serialization-benchmark/wiki/json-only

There are many failures. The original test was designed for full object serializers. If you think some of the testcases are too complex or contain stuff which cannot be done in JSon, let me now. I would then add new tests cases designed for JSon/Cross Platform encoding.

JSON-IO has the fewest failures but is slowest. Jackson seems very fast to me with databinding ..

— Reply to this email directly or view it on GitHub<

https://github.com/RuedigerMoeller/serialization-benchmark/issues/2#issuecomment-42741507>

.

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

— Reply to this email directly or view it on GitHub< https://github.com/RuedigerMoeller/serialization-benchmark/issues/2#issuecomment-42748591>

.

— Reply to this email directly or view it on GitHubhttps://github.com/RuedigerMoeller/serialization-benchmark/issues/2#issuecomment-42748886 .

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

[RichardHightower]

I have closed the distance by 1/2, and I have independent parse and independent serialize running as several times what Jackson is running. I have a String array mapping slow down on property map, but I went form 3 to 4x slower to only 2x slower. I think in a short while it will be faster.

The serialize and the parser are now faster. But the final map/injection into Java is slower.

[RichardHightower]

This is for StringPerformance. I have to focus one at a time.

[RuedigerMoeller]

I don't know if the samples are that representative for REST/JSon, they come from full binary serializers. And because native arrays are a lot faster than colelctions, we use them where possible. As said we can just add a typical JSOn testclass having a classic mix of some strings, ints, date and some collection. What I found strange is, that jackson can only handle hashmaps with String keys.

Most people first look on having no work, so its very important for you to not break. From my experience people just switch to another library without noticing you. That's why having a reproducible error is a gift, not a burden (however I can understand your ranting :-) ).

Regarding string performance: if you are fast at strings, than you are likely to be fast in benchmarks, strings are used a lot e.g. writing attribute names, classnames etc. . I don't think there is much room to get significant faster than jackson in raw json write performance (maybe Unsafe can give you some advantage). Regarding generic databinding/serialization there is some room for improvement. What I found with my FST (fastest generic java serialization on planet) is that adding some mechanical sympathy can give you an edge (lookout for L1 cache misses/high locality and branch prediction failures). Also eliminating object allocation whereever possible pays off (mostly because new objects tend to produce more L1 cache misses). Btw: does boon support restoring references (e.g. if an object is twice, only write it once ?). JSON IO does that.

[RichardHightower]

Jackson was 4x faster. It is now only 2x faster on my (not yet checked in version), and if it was not mother's day and I did not just get a new server that I wanted to play with (pixie botted my brand new i7 NUC to install KVM), and I did not deal with the GSON FUD, it would already be caught up.

It is not faster at parsing (JSON in). Just JSON out. Soon Jackson will not be faster at either. I will settle for a tie or 10% slower than Jackson. I will not settle for less. (I won't need to. I already have the parse and the encode faster. I just need to fix one mapping issue, and it is soup.)

Inline...

On Sun, May 11, 2014 at 8:19 AM, RuedigerMoeller notifications@github.comwrote:

I don't know if the samples are that representative for REST/JSon, they come from full binary serializers. And because native arrays are a lot faster than colelctions, we use them where possible.

Makes sense. I try to use char array when possible for parsing. :) Instead of some reader abstraction which I think is a common perf mistake.

As said we can just add a typical JSOn testclass having a classic mix of some strings, ints, date and some collection.

That is what I have and Boon handles it really well. REALLY WELL. :) But I want to handle arrays well too. So I don't mind. It is not the common case for JSON REST/Websocket (I have never seen it, but it is one that I would like to handle)

What I found strange is, that jackson can only handle hashmaps with String keys.

Boon can handle some weird stuff there. I know it handles enums as keys and primitives as keys. I did not realize Jackson could not handle that. (Is that case working for you?)

Most people first look on having no work, so its very important for you to not break.

From my experience people just switch to another library without noticing you.

I think JSON the objects are thought to be higher level.

That's why having a reproducible error is a gift, not a burden (however I can understand your ranting :-) ).

No. Honestly I just did not know what the 0ns means. I thought I invented some sort of JSON super computer so being an American, I contacted my lawyer and had it patented. :)

(In case that gets lost in translations.. Not by you, but by whoever reads this later. That was a joke.)

Regarding string performance: if you are fast at strings, than you are likely to be fast in benchmarks, strings are used a lot e.g. writing attribute names, classnames etc. .

:) That is not Boon's trouble. Key names, enums, etc. are fine. In fact ASCII range is fine too. It is unicode outside of ASCII range where Jackson soars. :)

Boon will catch up.

I don't think there is much room to get significant faster than jackson in raw json write performance (maybe Unsafe can give you some advantage).

I think I am faster. It is just unicode JSON encoding that I am having an issue with. I have a solutions in the works. I never knew it was an issue. I usually fix things PDQ once someone shows me. It was just missed in the benchmarks. We have unicode parse, and boon kicks butt. We were just missing this case, and I have some slower encode unicode JSON string stuff. I don't doubt that it can be fixed. :)

Like I said. You will see steady progress. One fix or so a week.

Regarding generic databinding/serialization there is some room for improvement.

Good to know.

What I found with my FST (fastest generic java serialization on planet) is that adding some mechanical sympathy can give you an edge (lookout for L1 cache misses/high locality and branch prediction failures).

How can I see this? What tool?

Also eliminating object allocation whereever possible pays off (mostly because new objects tend to produce more L1 cache misses).

I do try to do this. This has been where most performance gains come from and reducing if statements inside of for loops, but I still have a bunch. I am sure there is a lot of room for improvement here. I have added some tasks in github. I need to create a more generic set of mappers to reduce the number of if statements in tight loops.

Btw: does boon support restoring references (e.g. if an object is twice, only write it once ?).

Yes and no. Simple case is turned on my default. It will not write out an object twice, but it does not try to restore references ever. So it can avoid a circular dependency but it assumes client code can rewire references. The simple case of child referring to parent is always taken out of the serialization. Again for REST and Websocket, I think this makes sense. There is a more complex case it handles, but again I don't plan on supporting auto reference rewire. FST is a drop in replacement for OOS. Boon is never meant to be. It is a serializer for JSON but not for complex object graphs for serialization. The best I will do is call a method after for the client code to wire in references.

It also does not handle inner classes. I don't plan on it either. I have never seen it in a production app.

JSON IO does that.

Cool. I think Boon will not.

— Reply to this email directly or view it on GitHubhttps://github.com/RuedigerMoeller/serialization-benchmark/issues/2#issuecomment-42773308 .

Rick Hightower (415) 968-9037 Profile http://www.google.com/profiles/RichardHightower

[RuedigerMoeller]

Cache misses

Check the overseer library to detect that. From my experience "if" branches aren't that expensive, its mostly L1 cache (pointer chaising).

Restoring References

Good to know, so for JSon I'd probably create a set of tests without reference restoring required :-)

features in general (inner classes, ref restoring)

The easier it is for a programmer to get his stuff JSon'ed the more likely your stuff will succeed. Performance is rarely the only reason (except for stock JDK serialization, which is awful slow).

[RuedigerMoeller]

http://jpbempel.blogspot.de/2013_10_01_archive.html

[RichardHightower]

The cool thing about OS is learning stuff you don't run into at work.