Open GUI opened 3 months ago
Thank you for these numbers and the test file. I had a similar experience while working on updating the benchmarks in https://github.com/gjtorikian/commonmarker/pull/276/commits/0224ec83a46419c63d9e18ccf6004f5ca260cad9.
To be honest, I have not really looked into the performance of this gem, but I am starting to come around to it being the next thing to look at. I don't know when I'll have the time to dedicate to measuring the performance. I can say that it's pretty likely that any potential optimizations can be made in comrak; Commonmarker is just a really dumb wrapper around that lib.
I noticed that Commonmarker (v1.0.4) was performing worse than Kramdown for my own documents, which I wasn't expecting based on the benchmarks. I was able to replicate your benchmark results on my own computer (where commonmarker appears faster than Kramdown when using the default 11MB bencmark markdown file), but where I found a discrepancy was if I started benchmarking smaller documents, where then Commonmarker appears to be one of the slower options.
This slowdown for small documents is not the case for the v0.23.10 release of Commonmarker, so I think it's somehow tied to the switch to comrak. So I'm not sure if this is an issue with comrak or in the Ruby bindings, but I thought I'd start here since I stumbled into this when comparing Ruby libraries. I'm wondering if there's perhaps some overhead in initializing things or calling comrak that the large benchmark file maybe masks (since you'd be doing fewer iterations of calling Commanmarker repeatedly with a big document where more of the overhead is in the underlying Markdown parsing)? But I'm happy to shift this conversation over to comrak if you believe this is an issue on their end.
Here's what I'm seeing on my M1 MacBook Pro (observed both in Linux Docker images and directly on the Mac, but all of these tests would be using aarch64 binaries if that makes a difference):
v1.0.4
release with defaultbenchinput.md
input (11MB file)Roughly in line with the results from the readme where commonmarker is ~4x slower than redcarpet and Kramdown is ~60x slower than redcarpet.
v1.0.4
release withlorem1.md
input (3.7KB file)I used this lorem1.md sample file for a smaller benchmark file that is more representative of the files I'm passing through the library.
As you can see, with this smaller file Kramdown is now 44x slower than Redcarpet but Commonmarker is 131x slower than Redcarpet (and about 3x slower than Kramdown).
v0.23.10
release with defaultbenchinput.md
input (11MB file)I then switched over to the libcmark-gfm based release of this gem to see how these same test files performed on the benchmark suite in that branch:
With the large file Commonmarker is around 1.5x slower than Redcarpet. This also shows that Commonmarker v1.0.4 is about 2-3x slower than Commonmarker v0.23.10 for this type of large file, for whatever that's worth, but I realize you may have expected some performance differences with the switch in libraries.
v0.23.10
release withlorem1.md
input (3.7KB file)Here you can see the bigger difference between the old and new releases of Commonmarker, since Commonmarker is nearly as fast as Redcarpet with this small file on under v0.23.10, but significantly slower in v1.0.4.
Let me know if you have any questions, but thanks for all your work on this gem!