cowtowncoder
commented
6 years ago For what it is worth, some already exists:
- https://github.com/FasterXML/jackson-benchmarks (compares performance of jackson backends, used internally for verifying optimizations)
- https://github.com/FasterXML/jvm-json-benchmark (compares jackson json to other implementations, although not updated recently)
and then there is off-project jvm-serializers:
But the real problem in coming up with benchmarks is finding actually meaningful things to test.
Aside from bad methodology the next biggest flaw, I think, is that many benchmarks are toys, testing trivial things like decoding { "message" : "Hello World!" } or [1,2,3].
So getting somewhat more representative (of some real use case; and ideally matching data) use case, data set(s) would be useful.
This is not to say that adding benchmarks would not be useful. Just that the lack of more widely published benchmarks, results, is due to it being lots of work with relatively little benefit to author(s).
feinstein
I know some people frown upon benchmarks provided by the core team, but it's very hard to properly benchmark on Java, since the JVM needs proper warm-up and etc and most if not all the benchmarks on the internet comparing Jackson to GSON appears to be wrong (check the comments section, everyone of them have a valuable point about what's wrong).
The HikariCP project provides it's own benchmarks and explains on it's documentation about the right way to test the library. This is very important since public articles can spread misinformation just for not knowing the proper way to use the library, to take advantage of its optimizations (let alone how to benchmark in java or use the JMH).
Providing benchmarks on the library only adds to end information, if the readers want an external opinion, they can feel free to look for outside benchmarks if they want to, so no negative downsides to this, the worst thing that can happen is raising the level of external benchmarks.