Startup time of dojo CLI is high

[JamesLMilner]

Enhancement

At the moment the startup time for running dojo is around 2000+ milliseconds:

It looks like the majority of the time is spent loading modules.

Here's the profile as a ZIP: CPU-20190204T224612.cpuprofile.zip

Package Version: 5.0.1-pre

Code

dojo

Expected behavior:

The CLI loads snappily

Actual behavior:

There is a noticeable delay

[JamesLMilner]

I've created a branch that tries to get around this by caching the group command and description and then reusing them. This is good in that it reduces calling dojo dojo --help dojo --version from ~2000+ms to ~100ms on my machine. However this still isn't great because the first run will still be slow as the results are not cached yet. The caching logic also adds noise to the code and feels like a bit of a work around.

So, I have a wild suggestion. We change all dojo commands to have a command.json file with the following schema:

{
    "name": "command",
    "group": "group",
    "alias": ["c"],
    "description": "Some cool command"
}

This can be referenced from the command itself to fill in the current command structure. Doing this gives us a useful property that we can load it separately in the CLI with fs.readFileSync or fs.readFile to get the JSON without having to parse the whole module. We could then defer module loading in the run/eject'/register/validate calls so that the module is only loaded when it's needed. Reading the file is slow we could still use caching as per my example branch.

Overall I this should:

• Make CLI calls that don't run commands (dojo, dojo --help, dojo --version etc) feel instantaneous to the user (sub 100ms feels immediate)
• In theory make all command calls quicker because we don't load all command modules upfront (I think?)

Thoughts? @agubler @matt-gadd and any one else who's interested!

[JamesLMilner]

Speaking with @matt-gadd @agubler, maybe a good solution would be to add these things to the package.json of commands, rather than adding a new file. This would still require updating all commands to abide by this logic.

[schontz]

On my Windows 10 box it takes 15 seconds before running npm run dev starts showing the building animation. I have an i5 processor, SSD, and 24GB RAM, so I don't think my machine is to blame.

[schontz]

@JamesMilnerUK How did you create that profile? I can provide one for Windows if need be.

[JamesLMilner]

@schontz so first you'll want to find the location of the dojo script (for example whereis) then run node --inspect-brk /home/james/.nvm/versions/node/v8.2.1/bin/dojo where the path is your dojo script. You can then use the Chrome node debugger to create a flamegraph.

more info:

https://nodejs.org/en/docs/guides/debugging-getting-started/ https://medium.com/the-node-js-collection/debugging-node-js-with-google-chrome-4965b5f910f4

I haven't been able to reproduce this graph after looking at the performance again. It's coming in around ~450ms which is still not ideal but much faster than ~2000 I was getting before @matt-gadd @maier49

[schontz]

One of my projects is taking 5-6 seconds now. I tried it on another and it still took 15 seconds or so. I tried running a profile in Dev Tools but it crashed it. Fun times.

[evra]

May be this is not related to the @dojo/cli but rather to @dojo/cli-build-app. After upgrading to v5 we got huge performance issue with the dojo build -m dev -w memory -s -l. It takes up to 2 min to complete a build and about 40 - 50 sec. on reload.

I did some profiling and found out a possible reason. It's a globing pattern for the extra-watch-webpack-plugin. See https://github.com/dojo/cli-build-app/blob/1ee772282d606d41d073ad7b934c113f72e7b1e7/src/base.config.ts#L449

Currently it is set to files: ['!(output|.*|node_modules)/**'] After changing it to files: ['src/**'] i got 20 sec. build time on reload After changing it to dirs: [ 'src' ] i got 10 sec. build time on reload

[mwistrand]

@evra Thanks for taking the time to whittle that down!

[maier49]

Removing all imports not used by the build command did not significantly alter the build time, and on Windows had no effect on the time to display the "building" animation, so I don't think there will be any significant timesave from moving around when/where modules are imported.