privatenumber / minification-benchmarks

🏃‍♂️🏃‍♀️🏃 JS minification benchmarks: babel-minify, esbuild, terser, uglify-js, swc, google closure compiler, tdewolff/minify
MIT License
1.35k stars 31 forks source link

Document the fact that Closure advanced optimizations are disabled #598

Open adrianholovaty opened 1 month ago

adrianholovaty commented 1 month ago

Hello — this pull request documents the fact that these benchmarks disable Google Closure Compiler's "advanced optimizations." In my opinion this is important context for interpreting the results.

The specific context I intend to communicate in the pull request is:

If people are using these benchmarks to decide which minification library to use on their own JS projects, they should be aware of the Closure Compiler advanced optimizations possibility. Otherwise this set of benchmarks is misleading — or, at the very least, paints an incomplete picture.

I have no affiliation with Google or with the Closure Compiler project, aside from having used it for 10+ years now in my own projects. If you take the time to structure your code to take advantage of the "advanced optimizations," it results in significantly better minification than any other library.

Previously: issue #168 brought up the fact that these benchmarks don't use advanced optimizations, and the issue was closed. If the scope of these benchmarks is to remain on third-party JS libraries, I agree it makes sense not to do an advanced optimizations test — because that would require you to tweak those third-party JS libraries. But I do think you should be transparent about this limitation.

privatenumber commented 1 month ago

Thank you for the thoughtful input @adrianholovaty

My goal with this project is to evaluate JS minifiers on equal footing, using minimal/default configurations. I make an exception for Terser and Uglify as the "no compress" option is widely used in practice.

That said, other minifiers also have configurations and optimizations that I'm not utilizing. If we highlight that Google Closure Compiler's advanced optimizations aren't being used, we would also need to point out the other optimizations that aren't leveraged across all the minifiers.

To avoid opening that can of worms, I might prefer to update the Methodology section to clarify that these benchmarks don't demonstrate the full potential of each minifier and that they use default settings and process arbitrary third-party code untailored to any specific optimizations. We can also encourage users to learn more about each minifier to understand the full range of optimizations.

What do you think?

adrianholovaty commented 1 month ago

@privatenumber Thanks for the response. I definitely understand not wanting to open that can of worms. :) Your idea to update the Methodology section sounds like a positive move, for sure.