fabienrenaud / java-json-benchmark

Performance testing of serialization and deserialization of Java JSON libraries
MIT License
968 stars 134 forks source link

Add benchmarking for Jackson Blackbird #51

Closed darxriggs closed 1 year ago

darxriggs commented 1 year ago

Blackbird will eventually replace Afterburner.

see https://github.com/FasterXML/jackson-modules-base/tree/2.13/blackbird

NOTE: When running this on Java <9, the following warning is logged (incl. a stacktrace).

com.fasterxml.jackson.module.blackbird.util.ReflectionHack$Java9Up init WARNING: Unable to find Java 9+ MethodHandles.privateLookupIn. Blackbird is not performing optimally! java.lang.NoSuchMethodException: no such method: java.lang.invoke.MethodHandles.privateLookupIn(Class,Lookup)Lookup/invokeStatic

fabienrenaud commented 1 year ago

The code change LGTM

Please run the ser+deser benchmark with jackson blackbird (and another lib – "normal" jackson itself is fine) for both user and client objects and post the results here.

darxriggs commented 1 year ago

For Java 8 the performance is similar to not using Blackbird.

==> clients-deser.txt <==
Benchmark                             Mode  Cnt       Score       Error  Units
Deserialization.jackson              thrpt   20  752574.457 ± 16486.977  ops/s
Deserialization.jackson_afterburner  thrpt   20  795668.327 ± 11206.301  ops/s
Deserialization.jackson_blackbird    thrpt   20  761029.799 ±  8645.399  ops/s

==> clients-ser.txt <==
Benchmark                           Mode  Cnt        Score       Error  Units
Serialization.jackson              thrpt   20  1174292.971 ±  9646.695  ops/s
Serialization.jackson_afterburner  thrpt   20  1170286.398 ±  8048.048  ops/s
Serialization.jackson_blackbird    thrpt   20  1161072.743 ± 10761.879  ops/s

==> users-deser.txt <==
Benchmark                             Mode  Cnt       Score       Error  Units
Deserialization.jackson              thrpt   20  883583.420 ± 11052.714  ops/s
Deserialization.jackson_afterburner  thrpt   20  996068.962 ± 12927.690  ops/s
Deserialization.jackson_blackbird    thrpt   20  955941.604 ± 11175.956  ops/s

==> users-ser.txt <==
Benchmark                           Mode  Cnt        Score       Error  Units
Serialization.jackson              thrpt   20  1238316.459 ± 10419.201  ops/s
Serialization.jackson_afterburner  thrpt   20  1143120.847 ±  6203.100  ops/s
Serialization.jackson_blackbird    thrpt   20  1134477.650 ± 77062.931  ops/s

For Java 17 (where -XX:+AggressiveOpts is not available anymore) the optimization is more visible.

==> clients-deser.txt <==
Benchmark                             Mode  Cnt       Score       Error  Units
Deserialization.jackson              thrpt   20  682063.587 ± 27509.710  ops/s
Deserialization.jackson_afterburner  thrpt   20  778506.508 ± 14519.074  ops/s
Deserialization.jackson_blackbird    thrpt   20  726920.433 ± 25294.936  ops/s

==> clients-ser.txt <==
Benchmark                           Mode  Cnt        Score       Error  Units
Serialization.jackson              thrpt   20  1162670.014 ± 14400.889  ops/s
Serialization.jackson_afterburner  thrpt   20  1210346.495 ±  5986.547  ops/s
Serialization.jackson_blackbird    thrpt   20  1170983.688 ± 30142.461  ops/s

==> users-deser.txt <==
Benchmark                             Mode  Cnt       Score       Error  Units
Deserialization.jackson              thrpt   20  828565.647 ± 15561.984  ops/s
Deserialization.jackson_afterburner  thrpt   20  934245.752 ±  6494.479  ops/s
Deserialization.jackson_blackbird    thrpt   20  889663.712 ±  8100.982  ops/s

==> users-ser.txt <==
Benchmark                           Mode  Cnt        Score      Error  Units
Serialization.jackson              thrpt   20  1171206.755 ± 8770.593  ops/s
Serialization.jackson_afterburner  thrpt   20  1279912.496 ± 7519.745  ops/s
Serialization.jackson_blackbird    thrpt   20  1185730.807 ± 7631.782  ops/s

But all in all the variance between different runs is pretty big so comparisons are difficult.

fabienrenaud commented 1 year ago

Great! Thank you for your contribution @darxriggs