[pug-code-gen] Code generation performance

[jeromew]

From @joliss on January 18, 2017 15:54

I was surprised that code generation took such a large percentage of total execution time for Pug, so I took a quick look. Toying around with console.time and the profiler using an ad-hoc benchmark, it seems that there's a bunch of potential for optimization. Unfortunately I don't have the time right now to take a stab at actually optimizing it, but I wanted to record what I found for posterity.

• The overhead caused by compileDebug: true (the default) is about 40% of total (source). Maybe that's optimizable, or maybe there are cases where we know that we won't need the debug information.
• Much of the code generation time seems to be spent on this.buf.push. This jsPerf suggests that it may be quicker to use += to produce a giant string. Maybe there are even more possible optimizations along this line.
• In development mode, or when generating static pages: We may want to disable the Constantinople optimization. Constantinople takes up about 20% of the total time (source), with compileDebug: false.
• In development mode for server-side templates, or when generating static pages: It might be quicker to avoid code generation altogether and produce the output straight from the AST (or some intermediate representation).
• When you have a bunch of interdependent files (via include or extends) and you're rebuilding them repeatedly in development mode, we might be able to pre-generate some code for each individual file and cache it, so that we can do less work in the final pass that combines all the files.

Copied from original issue: pugjs/pug-code-gen#32

[jeromew]

From @ForbesLindesay on January 18, 2017 16:34

Disabling constantinople is something we could offer as an option. Unfortunately it's hard to automatically know when people are going to need it. Perhaps we could disable constantinople by default when people are using render/renderFile with no cache enabled.

There may well be options for optimising compileDebug. Most notably, https://github.com/pugjs/pug-code-gen/blob/1107c8c19d75285af63d7f4e94831efe01626493/index.js#L264-L270 is the bit I would expect to be costing us a lot of time. Perhaps we could cache the stringifyed filenames. Perhaps we could avoid outputting that debug information for nodes that we know won't throw errors. For example, if we keep using constaninople, then we can skip outputting debugging info for tags where all attributes are constant (or perhaps just where there are no attributes).

Another approach we could take, is to attempt rendering with compileDebug: false then re-compile and re-render if there are any errors. That would make us 40% (based on your measurements) faster in the case of a successful render, but quite a lot slower in the case of errors.

[jeromew]

From @joliss on January 18, 2017 17:51

Disabling constantinople is something we could offer as an option. Unfortunately it's hard to automatically know when people are going to need it.

I think just adding an option would be a big win. It seems to me that there are so many disparate use cases for Pug -- {server-side static, server-side dynamic, client-side dynamic} x {production, development, testing} -- that it's best to just expose all the options and let the tooling that calls Pug determine what combination is best. For example, if a tool like Ember CLI were to use Pug, that kind of tool will generally "know" which set of options is appropriate and call Pug accordingly.

[joliss]

On the matter of debugging output (compileDebug: true), it seems that there is a bunch of potential for optimization by simply making the output shorter. Let's say we run:

console.log(require('./packages/pug').compileClient('div foo', {
  filename: '/home/ubuntu/tmp/pugtest/foo.pug',
  inlineRuntimeFunctions: false
}))

and run the resulting string through a beautifier. Then we get:

function template(locals) {
  var pug_html = "",
    pug_mixins = {},
    pug_interp;
  var pug_debug_filename, pug_debug_line;
  try {;
    pug_debug_line = 1;
    pug_debug_filename = "\u002Fhome\u002Fubuntu\u002Ftmp\u002Fpugtest\u002Ffoo.pug";
    pug_html = pug_html + "\u003Cdiv\u003E";;
    pug_debug_line = 1;
    pug_debug_filename = "\u002Fhome\u002Fubuntu\u002Ftmp\u002Fpugtest\u002Ffoo.pug";
    pug_html = pug_html + "foo\u003C\u002Fdiv\u003E";
  } catch (err) {
    pug.rethrow(err, pug_debug_filename, pug_debug_line);
  };
  return pug_html;
}

It's easy to think of a bunch of ways to make the output shorter, but most notably pug_debug_filename and pug_debug_line are being reassigned repeatedly even when they don't change.

[ForbesLindesay]

This would be good to do. It's non-trivial to know when this does and does not need to be written out, but it's definitely worth doing.

[ajimix]

Any progress on this matter? Rendering bigger or complex templates is slow with pug

[janoside]

Any update here? As templates have gotten more complex, I'm getting to the point that performance in "production" is getting bad enough that I feel the need to consider alternative template engines.

Is this project languishing/dying?