fabienrenaud / java-json-benchmark

Performance testing of serialization and deserialization of Java JSON libraries
MIT License
973 stars 134 forks source link

Add benchmarking for qson #52

Closed darxriggs closed 1 year ago

darxriggs commented 2 years ago

see https://github.com/quarkusio/qson (this implements #43)

The library only supports access via getters & setters. The library does not support arrays, therefore the "Clients" tests are disabled.

BUT: With these, other libraries start to fail. (This has been solved by changing the fields from public to private.)

@fabienrenaud Did you encounter similar problems with getters & setters in the past when implementing the benchmarks or have a suggestion how to handle the mentioned issue?

fabienrenaud commented 2 years ago

I didn't encounter such issues within this project. Which libraries in particular are failing? AFAIR, you might have to tune each library to not use getters/setters when available and/or use lib annotations on each getter/setter to disable their usage.

darxriggs commented 2 years ago

The tests for avajeJsonb_diesel and jsonsmart fail and the one for gson hangs.

fabienrenaud commented 2 years ago

@darxriggs If any libs are failing, make sure upgrade them to their latest version and file bugs with lib maintainers if they are still failing.

If testing qson requires creating setters & getters, fine... In that case, let's make all class attributes private. I originally made them public to reduce the verbosity of these classes and because I'd expect simple public getter/setters to be inlined by the hotspot JVM and amount to the same as use of public attributes... The concern with having both public field and getters+setters is that:

  1. It is no longer a realistic example – Why write a class with both public attributes and their public getters/setters that do nothing more?
  2. It becomes unclear what path libs use to set/get attributes. Perhaps some prefer public attributes, perhaps some prefer getters/setters. Either way, because of point 1, keeping both ways would no longer be an apple-to-apple comparison.

IMO a bug should also be filed with qson to support the case of public attributes properly since setting/reading from public fields doesn't require reflection... The lack of support for arrays also seems to indicate this lib is highly specialized and not an all purpose JSON lib, which are the primary type of libs intended to be tested here...

darxriggs commented 2 years ago

After changing the fields from public to private and adding getters & settters, the problems with other libraries are resolved.

The benchmarks with qson are as follows.

$ ./run -t 8 deser --apis databind --libs jackson,qson
Benchmark                             Mode  Cnt       Score       Error  Units
Deserialization.jackson              thrpt   20  831149.378 ± 11810.192  ops/s
Deserialization.jackson_afterburner  thrpt   20  962902.099 ± 21800.463  ops/s
Deserialization.jackson_blackbird    thrpt   20  893973.038 ±  4591.974  ops/s
Deserialization.qson                 thrpt   20  693358.092 ±  5774.933  ops/s
$ ./run -t 8 ser --apis databind --libs jackson,qson
Benchmark                           Mode  Cnt        Score       Error  Units
Serialization.jackson              thrpt   20  1190688.659 ± 15423.080  ops/s
Serialization.jackson_afterburner  thrpt   20  1219402.057 ± 65896.171  ops/s
Serialization.jackson_blackbird    thrpt   20  1167038.202 ± 78403.318  ops/s
Serialization.qson                 thrpt   20   748794.359 ±  4530.767  ops/s
darxriggs commented 1 year ago

@fabienrenaud ping

fabienrenaud commented 1 year ago

Thanks!

darxriggs commented 1 year ago

Thank you for maintaining this repository!