Benchmarks should precompile templates

[benasher44]

Is compilation a useful metric when thinking about the big picture? Most/all of these templating engines allow precompiling templates. Compiling and reusing the compiled result is best practice, well-documented, and results in better performance for all of the other engines.

It's great that eta has achieved such great compilation performance, but once you realize how these benchmarks work (compiling each time through the loop), it makes the benchmarks not that useful—kinda waters them down. I think most of the time you care about render performance. How and when you compile the template is a detail I think most can sort out, in order to get good render performance.

It'd be great to precompile for all of the templating engines for these benchmarks to align them with real-world usage, or at minimum include that as a supplementary benchmark to what you have now.

[benasher44]

Here's an updated benchmark where templates are precompiled: https://ghcdn.rawgit.org/benasher44/eta/fdb05f6229765388965bce7626d06b5daed65b77/browser-tests/benchmark.html. Eta is still quite fast, but handlebars, for example, is 8x (on my machine) improved by precompiling the template.

[benasher44]

Happy to open a PR with the precompiling version, if that'd be helpful!

[shadowtime2000]

@benasher44 Nice take! I'm completely down for a PR with a benchmark where templates are precompiled.

[benasher44]

@shadowtime2000 great thanks! Can I replace the current benchmarks? I think precompiled version is more practical. It will mean that benchmarks no longer show eta as "1st place", but I think it's a better north star, as it's more representative of real usage.

[shadowtime2000]

@benasher44 Yes, that would be great!

Edit: btw assigning to you for organizational purposes

[benasher44]

Great thanks!