Closed feinstein closed 6 years ago
For what it is worth, some already exists:
and then there is off-project jvm-serializers
:
But the real problem in coming up with benchmarks is finding actually meaningful things to test.
Aside from bad methodology the next biggest flaw, I think, is that many benchmarks are toys, testing trivial things like decoding { "message" : "Hello World!" }
or [1,2,3]
.
So getting somewhat more representative (of some real use case; and ideally matching data) use case, data set(s) would be useful.
This is not to say that adding benchmarks would not be useful. Just that the lack of more widely published benchmarks, results, is due to it being lots of work with relatively little benefit to author(s).
I understand and agree that it's lots of work with relatively little benefit to author(s)
... But the benefits to the users are very tangible, and for me that's one more reason to include it on the documentations.
Unfortunately, I am not experienced enough to provide real use case JSON scenarios, but I believe most major companies using Jackson will be willing to contribute with anonymous JSON data thats meaningful and representative of their work. Is it hard to approach them?
@feinstein I do not have time to pro-actively go and ask for data. This is hobby for me, not job. I help whenever opportunity exposes itself but at the end of the day overworking myself to only help others and not me helps no one.
@cowtowncoder I understand, IMHO I don't think asking for data is that much complicated, and if I was the creator of Jackson I would be curious about it, but I understand your point and your goals.
Yes, I think we agree there: it would be great to get more data, tests. Would it be possible for you to file this at https://github.com/FasterXML/jackson-future-ideas/issues ? I would move it but I don't think github has transfer issue functionality (alas)?
I'll also send a note on mailing lists in case someone knows of open data sources for json.
@cowtowncoder done. Thank you for your time :)
Thanks!
I know some people frown upon benchmarks provided by the core team, but it's very hard to properly benchmark on Java, since the JVM needs proper warm-up and etc and most if not all the benchmarks on the internet comparing Jackson to GSON appears to be wrong (check the comments section, everyone of them have a valuable point about what's wrong).
The HikariCP project provides it's own benchmarks and explains on it's documentation about the right way to test the library. This is very important since public articles can spread misinformation just for not knowing the proper way to use the library, to take advantage of its optimizations (let alone how to benchmark in java or use the JMH).
Providing benchmarks on the library only adds to end information, if the readers want an external opinion, they can feel free to look for outside benchmarks if they want to, so no negative downsides to this, the worst thing that can happen is raising the level of external benchmarks.