Slow Start Times

[stephenplusplus]

The ever-haunting issue with yo is the slow discovery of installed generators. I recall us getting it in check in the past, but has anyone else noticed it creeping up again?

[sindresorhus]

I'm on a superfast Mac with SSD and it takes ~1s to startup, which is pretty slow.

[sindresorhus]

https://www.npmjs.org/package/time-require would be useful for debugging and profiling this.

[dickeylth]

I used time-require & it shows:

Start time: (2016-05-12 08:11:21 UTC) [treshold=1%]

module time %

1 chalk (../../gener...s/chalk/index.js) 22ms ▇ 1% 2 lodash (../../gene.../lodash/index.js) 42ms ▇ 2% 3 mem-fs (../../gene.../mem-fs/index.js) 27ms ▇ 1% 4 lodash (../../gene...lodash/lodash.js) 36ms ▇ 2% 5 ./baseUI (../../ge...lib/ui/baseUI.js) 40ms ▇ 2% 6 ./ui/bottom-bar (....ui/bottom-bar.js) 43ms ▇ 2% 7 inquirer (../../ge.../lib/inquirer.js) 99ms ▇▇ 5% 8 ./adapter (../../g...t/lib/adapter.js) 116ms ▇▇▇ 5% 9 yeoman-environment...b/environment.js) 234ms ▇▇▇▇▇ 11% 10 underscore.string....string/index.js) 39ms ▇ 2% 11 read-pkg (../../ge...ead-pkg/index.js) 42ms ▇ 2% 12 read-pkg-up (../.....-pkg-up/index.js) 43ms ▇ 2% 13 ast-query (../../g...uery/lib/tree.js) 44ms ▇ 2% 14 gruntfile-editor (...-editor/index.js) 45ms ▇ 2% 15 mem-fs-editor (../...-editor/index.js) 43ms ▇ 2% 16 ./api/attributes (...pi/attributes.js) 23ms ▇ 1% 17 ./api/traversing (...pi/traversing.js) 24ms ▇ 1% 18 ./lib/cheerio (../...o/lib/cheerio.js) 63ms ▇▇ 3% 19 cheerio (../../gen...cheerio/index.js) 64ms ▇▇ 3% 20 html-wiring (../.....-wiring/index.js) 66ms ▇▇ 3% 21 ./utils (../../gen...tch/lib/utils.js) 28ms ▇ 1% 22 ./lib/expand (../....ch/lib/expand.js) 31ms ▇ 1% 23 micromatch (../../...romatch/index.js) 32ms ▇ 2% 24 glob-stream (../.....-stream/index.js) 66ms ▇▇ 3% 25 ./lib/src (../../g...lib/src/index.js) 89ms ▇▇ 4% 26 vinyl-fs (../../ge...inyl-fs/index.js) 103ms ▇▇ 5% 27 decompress-tar (.....ess-tar/index.js) 25ms ▇ 1% 28 decompress-tarbz2...-tarbz2/index.js) 29ms ▇ 1% 29 decompress-targz (...s-targz/index.js) 31ms ▇ 1% 30 decompress (../../...ompress/index.js) 211ms ▇▇▇▇ 10% 31 gulp-util (../../g...lp-util/index.js) 39ms ▇ 2% 32 gulp-decompress (....ompress/index.js) 253ms ▇▇▇▇▇ 12% 33 download (../../ge...ownload/index.js) 296ms ▇▇▇▇▇▇ 14% 34 ./actions/fetch (....actions/fetch.js) 296ms ▇▇▇▇▇▇ 14% 35 got (../../generat...les/got/index.js) 30ms ▇ 1% 36 gh-got (../../gene.../gh-got/index.js) 31ms ▇ 1% 37 github-username (....sername/index.js) 31ms ▇ 1% 38 ./actions/user (...../actions/user.js) 45ms ▇ 2% 39 ./base (../../gene...ator/lib/base.js) 637ms ▇▇▇▇▇▇▇▇▇▇▇▇ 30% 40 yeoman-generator (...tor/lib/index.js) 876ms ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 41% 41 npm-updater (../.....updater/index.js) 26ms ▇ 1% 42 ../_libs/logo (../...am/_libs/logo.js) 34ms ▇ 2% yeoman-generator really become very slow than before.

• MacBook Pro (Retina, 13-inch, Early 2013)
• 2.6 GHz Intel Core i5
• 8 GB 1600 MHz DDR3
• npm@2.14.7
• node@4.4.3 (/usr/local/bin/node)
• darwin x64 15.4.0

[sindresorhus]

@dickeylth I would recommend upgrading to Node.js 6. They improved the require performance considerably in that version ;)

[dickeylth]

@sindresorhus unfortunately my local node tools are still not ready for npm 3, and npm3 itself still has so many big-bug issues to be resolved, so right now upgrading to Node.js v6 is still not a possible choice.😞

[Tiberriver256]

Still pretty darn slow these days... npm 6, node v8

[Olian04]

This issue is still very much relevant. I'm on Node v10.5.0, npm v6.1.0 and yo v2.0.3 and its taking up towards 10 minutes between calling yo foo and the first line of code being executed in foo.

[SBoudrias]

@Olian04 this is really abnormal. When we were talking about this, it was about lowering the startup time from ~1-2 seconds to near instant. Not minutes 😱

I'd be happy to help you figure out what's wrong. Try running DEBUG=yeoman:* yo and see if you can see where the time is spent.

[Olian04]

@SBoudrias welp, thats some difference.... :/ I'm not currently in the exact same environment (I've downgraded node to LTS) but it still takes ages to run. Since I'm on windows 10 i assumed DEBUG=yeoman:* meant Set-Variable -Name "DEBUG" -Value "yeoman:*". With that being set, just running yo and nothing else took a bit over 3 min before i hit the option screen. Am I supposed to look for a log file in some dir now, or did I set the variable the wrong way?

> yo

? 'Allo

! What would you like to do? Get me out of here!

     _-----_     ╭───────────────────────╮
    |       |    │      Bye from us!     │
    |--(o)--|    │       Chat soon.      │
   `---------´   │      Yeoman team      │
    ( _´U`_ )    │    http://yeoman.io   │
    /___A___\   /╰───────────────────────╯
     |  ~  |
   __'.___.'__
 ´   `  |° ´ Y `

[SBoudrias]

The Set-Variable probably didn't work. You should see a lot of internal yeoman logs being printed out. Try set DEBUG=yeoman:generator (http://yeoman.io/authoring/debugging.html)

[Olian04]

@SBoudrias I tried running set DEBUG=yeoman:generator and then yo but it didn't print anything extra. Would it make a difference if yo is installed globally or locally?

[Tiberriver256]

@SBoudrias - I'm in the same boat as @Olian04 too. Might be a Windows 10 thing? I've got minutes before yo comes back with a list of generators. Same time if I specify a specific generator.

@Olian04 from PowerShell it's an Environment variable. Try using this:

$Env:DEBUG = "yeoman:*"

Here is my response. Most of the load time is before the yeoman guy shows up and then a good chunk before the logging starts too. After the log messages start showing up it's pretty quick.

PS C:\Users\wmrm> $ENV:DEBUG = "yeoman:*"
PS C:\Users\wmrm> yo

     _-----_
    |       |    ╭──────────────────────────────────────────╮
    |--(o)--|    │ Update available: 2.0.3 (current: 2.0.2) │
   `---------´   │     Run npm install -g yo to update.     │
    ( _´U`_ )    ╰──────────────────────────────────────────╯
    /___A___\   /
     |  ~  |
   __'.___.'__
 ´   `  |° ´ Y `

  yeoman:environment found C:/Users/wmrm/AppData/Roaming/npm/node_modules/generator-generator/app/index.js, trying to register +0ms
  yeoman:environment Resolve namespaces for C:/Users/wmrm/AppData/Roaming/npm/node_modules/generator-generator/app/index.js: generator:app +0ms
  yeoman:environment Registered generator:app (C:\Users\wmrm\AppData\Roaming\npm\node_modules\generator-generator\app\index.js) +3ms
  yeoman:environment found C:/Users/wmrm/AppData/Roaming/npm/node_modules/generator-generator/subgenerator/index.js, trying to register +12ms
  yeoman:environment Resolve namespaces for C:/Users/wmrm/AppData/Roaming/npm/node_modules/generator-generator/subgenerator/index.js: generator:subgenerator +4ms
  yeoman:environment Registered generator:subgenerator (C:\Users\wmrm\AppData\Roaming\npm\node_modules\generator-generator\subgenerator\index.js) +2ms
  yeoman:environment found C:/Users/wmrm/AppData/Roaming/npm/node_modules/generator-haworthapp/app/index.js, trying to register +7s
  yeoman:environment Resolve namespaces for C:/Users/wmrm/AppData/Roaming/npm/node_modules/generator-haworthapp/app/index.js: haworthapp:app +7s
  yeoman:environment Registered haworthapp:app (C:\my\Generator-HaworthApp\app\index.js) +2ms
  yeoman:environment found C:/Users/wmrm/AppData/Roaming/npm/node_modules/generator-typescript-generator/app/index.js, trying to register +3s
  yeoman:environment Resolve namespaces for C:/Users/wmrm/AppData/Roaming/npm/node_modules/generator-typescript-generator/app/index.js: typescript-generator:app +3s
  yeoman:environment Registered typescript-generator:app (C:\Users\wmrm\AppData\Roaming\npm\node_modules\generator-typescript-generator\app\index.js) +2ms
  yeoman:environment found C:/Users/wmrm/AppData/Roaming/npm/node_modules/generator-node-typescript/generators/app/index.js, trying to register +36ms
  yeoman:environment Resolve namespaces for C:/Users/wmrm/AppData/Roaming/npm/node_modules/generator-node-typescript/generators/app/index.js: node-typescript:app +34ms
  yeoman:environment Registered node-typescript:app (C:\Users\wmrm\AppData\Roaming\npm\node_modules\generator-node-typescript\generators\app\index.js) +2ms
  yeoman:environment found C:/Users/wmrm/AppData/Roaming/npm/node_modules/generator-node-typescript/generators/classlib/index.js, trying to register +11ms
  yeoman:environment Resolve namespaces for C:/Users/wmrm/AppData/Roaming/npm/node_modules/generator-node-typescript/generators/classlib/index.js: node-typescript:classlib +9ms
  yeoman:environment Registered node-typescript:classlib (C:\Users\wmrm\AppData\Roaming\npm\node_modules\generator-node-typescript\generators\classlib\index.js) +2ms
? 'Allo Micah! What would you like to do? (Use arrow keys)

[Tiberriver256]

Massive improvement upgrading to 2.0.3 but still I think slower than would be expected. Almost 2 minutes. here is my time-require:

Start time: (2018-07-07 03:37:03 UTC) [treshold=1%]
 #  module                                        time  %
 1  rxjs/Rx (..\..\Users\..._modules\rxjs\Rx.js)    1s  ■ 1%
 2  ./ui/prompt (..\..\Us...er\lib\ui\prompt.js)    1s  ■ 1%
 3  inquirer (..\..\Users...rer\lib\inquirer.js)  1.2s  ■ 1%
 4  insight (..\..\Users\...nsight\lib\index.js)  1.3s  ■ 1%
 5  rxjs/Rx (..\..\Users\..._modules\rxjs\Rx.js)  1.1s  ■ 1%
 6  ./ui/prompt (..\..\Us...er\lib\ui\prompt.js)  1.1s  ■ 1%
 7  inquirer (..\..\Users...rer\lib\inquirer.js)  1.2s  ■ 1%
 8  ./adapter (..\..\User...ment\lib\adapter.js)  1.3s  ■ 1%
 9  yeoman-environment (....\lib\environment.js)  2.3s  ■ 2%
10  ./router (..\..\Users...es\yo\lib\router.js)  2.3s  ■■ 2%
Total require(): 4724
Total time: 1m 40.9s

     _-----_     ╭───────────────────────╮
    |       |    │      Bye from us!     │
    |--(o)--|    │       Chat soon.      │
   `---------´   │      Yeoman team      │
    ( _´U`_ )    │    http://yeoman.io   │
    /___A___\   /╰───────────────────────╯
     |  ~  |
   __'.___.'__
 ´   `  |° ´ Y `

[SBoudrias]

@Olian04 @Tiberriver256 maybe some inspiration on https://github.com/gulpjs/gulp/issues/1578 - looks like some windows users were having SSD disk issues which impacted node require() performance. 2.3s second import time isn't usual - but there's not much we can do on our side to fix that.

Theoretically, as yo is a top level dependency, I guess we could do some amount of bundling. But to me it's really unclear how much improvement this would end up giving people in your situation:

1. If we bundle only the yo core, then we remove ~10 imports maybe, potentially saving 10 seconds in your case? But that's not really gonna have a big impact as it'd still takes minutes.
2. If we bundle everything, then fixes/updates to sub-dependencies aren't reflected unless we release new yo versions. For example, we regularly push updates to yeoman-environment to make sure our support of new npm and yarn version is kept up to date.

I'm open to suggestion.

[Olian04]

@Tiberriver256 thanks for the advice I'll have a go with it once I get back to work on Monday.

@SBoudrias It might be worth mentioning that I've tried yo@2.0.3 & yo@2.0.1 both are as slow. However yo@1.8.5 is working as intended, so that's what I've been using previously.

[Olian04]

@SBoudrias I tried what @Tiberriver256 suggested, and this time it worked. However almost all of the time is spent before the first log message is printed.

PS C:\Users\xxqqvc\Projects\test> $Env:DEBUG = "yeoman:*"
PS C:\Users\xxqqvc\Projects\test> npx yo
npx: installed 1 in 2.19s
Path must be a string. Received undefined
C:\Users\xxqqvc\Projects\test\node_modules\yo\lib\cli.js
  yeoman:environment found C:/Users/claxxqqvc/AppData/Roaming/npm/node_modules/generator-markstache/generators/app/index.js, trying to register +0ms
  yeoman:environment Resolve namespaces for C:/Users/claxxqqvc/AppData/Roaming/npm/node_modules/generator-markstache/generators/app/index.js: markstache:app +0ms
  yeoman:environment Registered markstache:app (C:\Users\xxqqvc\Projects\generator-markstache\generators\app\index.js) +1ms
  yeoman:environment found C:/Users/claxxqqvc/AppData/Roaming/npm/node_modules/generator-microservice/generators/app/index.js, trying to register +30ms
  yeoman:environment Resolve namespaces for C:/Users/claxxqqvc/AppData/Roaming/npm/node_modules/generator-microservice/generators/app/index.js: microservice:app +24ms
  yeoman:environment Registered microservice:app (C:\Users\xxqqvc\Projects\generator-microservice\generators\app\index.js) +1ms
  yeoman:environment found C:/Users/claxxqqvc/AppData/Roaming/npm/node_modules/generator-microservice/generators/common-create/index.js, trying to register +2ms
  yeoman:environment Resolve namespaces for C:/Users/claxxqqvc/AppData/Roaming/npm/node_modules/generator-microservice/generators/common-create/index.js: microservice:common-create +2ms
  yeoman:environment Registered microservice:common-create (C:\Users\xxqqvc\Projects\generator-microservice\generators\common-create\index.js) +0ms
  yeoman:environment found C:/Users/claxxqqvc/AppData/Roaming/npm/node_modules/generator-microservice/generators/common-update/index.js, trying to register +2ms
  yeoman:environment Resolve namespaces for C:/Users/claxxqqvc/AppData/Roaming/npm/node_modules/generator-microservice/generators/common-update/index.js: microservice:common-update +1ms
  yeoman:environment Registered microservice:common-update (C:\Users\xxqqvc\Projects\generator-microservice\generators\common-update\index.js) +1ms
  yeoman:environment found C:/Users/claxxqqvc/AppData/Roaming/npm/node_modules/generator-microservice/generators/common-update-aurora/index.js, trying to register +3ms
  yeoman:environment Resolve namespaces for C:/Users/claxxqqvc/AppData/Roaming/npm/node_modules/generator-microservice/generators/common-update-aurora/index.js: microservice:common-update-aurora +2ms
  yeoman:environment Registered microservice:common-update-aurora (C:\Users\xxqqvc\Projects\generator-microservice\generators\common-update-aurora\index.js) +1ms
  yeoman:environment found C:/Users/claxxqqvc/AppData/Roaming/npm/node_modules/generator-microservice/generators/common-update-public/index.js, trying to register +2ms
  yeoman:environment Resolve namespaces for C:/Users/claxxqqvc/AppData/Roaming/npm/node_modules/generator-microservice/generators/common-update-public/index.js: microservice:common-update-public +1ms
  yeoman:environment Registered microservice:common-update-public (C:\Users\xxqqvc\Projects\generator-microservice\generators\common-update-public\index.js) +2ms
  yeoman:environment found C:/Users/claxxqqvc/AppData/Roaming/npm/node_modules/generator-microservice/generators/core-create/index.js, trying to register +3ms
  yeoman:environment Resolve namespaces for C:/Users/claxxqqvc/AppData/Roaming/npm/node_modules/generator-microservice/generators/core-create/index.js: microservice:core-create +2ms
  yeoman:environment Registered microservice:core-create (C:\Users\xxqqvc\Projects\generator-microservice\generators\core-create\index.js) +1ms
  yeoman:environment found C:/Users/claxxqqvc/AppData/Roaming/npm/node_modules/generator-microservice/generators/core-update/index.js, trying to register +3ms
  yeoman:environment Resolve namespaces for C:/Users/claxxqqvc/AppData/Roaming/npm/node_modules/generator-microservice/generators/core-update/index.js: microservice:core-update +1ms
  yeoman:environment Registered microservice:core-update (C:\Users\xxqqvc\Projects\generator-microservice\generators\core-update\index.js) +1ms
  yeoman:environment found C:/Users/claxxqqvc/AppData/Roaming/npm/node_modules/generator-microservice/generators/java-create/index.js, trying to register +2ms
  yeoman:environment Resolve namespaces for C:/Users/claxxqqvc/AppData/Roaming/npm/node_modules/generator-microservice/generators/java-create/index.js: microservice:java-create +2ms
  yeoman:environment Registered microservice:java-create (C:\Users\xxqqvc\Projects\generator-microservice\generators\java-create\index.js) +0ms
  yeoman:environment found C:/Users/claxxqqvc/AppData/Roaming/npm/node_modules/generator-microservice/generators/java-update/index.js, trying to register +2ms
  yeoman:environment Resolve namespaces for C:/Users/claxxqqvc/AppData/Roaming/npm/node_modules/generator-microservice/generators/java-update/index.js: microservice:java-update +2ms
  yeoman:environment Registered microservice:java-update (C:\Users\xxqqvc\Projects\generator-microservice\generators\java-update\index.js) +0ms
  yeoman:environment found C:/Users/xxqqvc/Projects/generator-markstache/generators/app/index.js, trying to register +3ms
  yeoman:environment Resolve namespaces for C:/Users/xxqqvc/Projects/generator-markstache/generators/app/index.js: markstache:app +3ms  yeoman:environment Registered markstache:app (C:\Users\xxqqvc\Projects\generator-markstache\generators\app\index.js) +0ms
  yeoman:environment found C:/Users/xxqqvc/Projects/generator-microservice/generators/app/index.js, trying to register +10ms
  yeoman:environment Resolve namespaces for C:/Users/xxqqvc/Projects/generator-microservice/generators/app/index.js: microservice:app +9ms
  yeoman:environment Registered microservice:app (C:\Users\xxqqvc\Projects\generator-microservice\generators\app\index.js) +0ms
  yeoman:environment found C:/Users/xxqqvc/Projects/generator-microservice/generators/common-create/index.js, trying to register +1ms
  yeoman:environment Resolve namespaces for C:/Users/xxqqvc/Projects/generator-microservice/generators/common-create/index.js: microservice:common-create +2ms
  yeoman:environment Registered microservice:common-create (C:\Users\xxqqvc\Projects\generator-microservice\generators\common-create\index.js) +0ms
  yeoman:environment found C:/Users/xxqqvc/Projects/generator-microservice/generators/common-update/index.js, trying to register +2ms
  yeoman:environment Resolve namespaces for C:/Users/xxqqvc/Projects/generator-microservice/generators/common-update/index.js: microservice:common-update +2ms
  yeoman:environment Registered microservice:common-update (C:\Users\xxqqvc\Projects\generator-microservice\generators\common-update\index.js) +1ms
  yeoman:environment found C:/Users/xxqqvc/Projects/generator-microservice/generators/common-update-aurora/index.js, trying to register +3ms
  yeoman:environment Resolve namespaces for C:/Users/xxqqvc/Projects/generator-microservice/generators/common-update-aurora/index.js: microservice:common-update-aurora +1ms
  yeoman:environment Registered microservice:common-update-aurora (C:\Users\xxqqvc\Projects\generator-microservice\generators\common-update-aurora\index.js) +1ms
  yeoman:environment found C:/Users/xxqqvc/Projects/generator-microservice/generators/common-update-public/index.js, trying to register +2ms
  yeoman:environment Resolve namespaces for C:/Users/xxqqvc/Projects/generator-microservice/generators/common-update-public/index.js: microservice:common-update-public +1ms
  yeoman:environment Registered microservice:common-update-public (C:\Users\xxqqvc\Projects\generator-microservice\generators\common-update-public\index.js) +1ms
  yeoman:environment found C:/Users/xxqqvc/Projects/generator-microservice/generators/core-create/index.js, trying to register +1ms
  yeoman:environment Resolve namespaces for C:/Users/xxqqvc/Projects/generator-microservice/generators/core-create/index.js: microservice:core-create +1ms
  yeoman:environment Registered microservice:core-create (C:\Users\xxqqvc\Projects\generator-microservice\generators\core-create\index.js) +0ms
  yeoman:environment found C:/Users/xxqqvc/Projects/generator-microservice/generators/core-update/index.js, trying to register +2ms
  yeoman:environment Resolve namespaces for C:/Users/xxqqvc/Projects/generator-microservice/generators/core-update/index.js: microservice:core-update +1ms
  yeoman:environment Registered microservice:core-update (C:\Users\xxqqvc\Projects\generator-microservice\generators\core-update\index.js) +1ms
  yeoman:environment found C:/Users/xxqqvc/Projects/generator-microservice/generators/java-create/index.js, trying to register +1ms
  yeoman:environment Resolve namespaces for C:/Users/xxqqvc/Projects/generator-microservice/generators/java-create/index.js: microservice:java-create +1ms
  yeoman:environment Registered microservice:java-create (C:\Users\xxqqvc\Projects\generator-microservice\generators\java-create\index.js) +0ms
  yeoman:environment found C:/Users/xxqqvc/Projects/generator-microservice/generators/java-update/index.js, trying to register +2ms
  yeoman:environment Resolve namespaces for C:/Users/xxqqvc/Projects/generator-microservice/generators/java-update/index.js: microservice:java-update +2ms
  yeoman:environment Registered microservice:java-update (C:\Users\xxqqvc\Projects\generator-microservice\generators\java-update\index.js) +0ms
  yeoman:environment found C:/Users/xxqqvc/Projects/test/node_modules/generator-git-migrator/generators/app/index.js, trying to register +2ms
  yeoman:environment Resolve namespaces for C:/Users/xxqqvc/Projects/test/node_modules/generator-git-migrator/generators/app/index.js: git-migrator:app +2ms
  yeoman:environment Registered git-migrator:app (C:\Users\xxqqvc\Projects\test\node_modules\generator-git-migrator\generators\app\index.js) +1ms
? 'Allo

! What would you like to do? Get me out of here!

     _-----_     ╭───────────────────────╮
    |       |    │      Bye from us!     │
    |--(o)--|    │       Chat soon.      │
   `---------´   │      Yeoman team      │
    ( _´U`_ )    │    http://yeoman.io   │
    /___A___\   /╰───────────────────────╯
     |  ~  |
   __'.___.'__
 ´   `  |° ´ Y `

PS C:\Users\xxqqvc\Projects\test>

[MarcoScabbiolo]

Regarding how global-tunnel-ng might have affected the speed of yo, the package itself runs smoothly, 95 tests take less than 2 seconds to complete. If running yo behind a proxy any network request will take a little bit longer, how much will obviously depend on the network conditions.

I've tested the latest version of yo behind a corporate proxy running generator-node and also listing generators and it took less than 2 seconds for every operation using both environment variables and the npm configuration to set the proxy.

Are you doing anything in particular to cause these delays?

[Olian04]

@MarcoScabbiolo not to my knowledge. The proxy doesn't slow down any of my other tools. Also, why would the proxy even factor in when it takes minutes between running yo and the first line being printed?

[MarcoScabbiolo]

@Olian04 I've reproduced your environment (Win 10, Node LTS 8.11.3) and I have a 5-6 seconds delay. Using WSL (Ubuntu) the delay is 1-2 seconds.

All CLIs in Windows are very slow, PowerShell is not an exception. It is also usual for CLIs in Windows to get stuck while running any command, and you need to Ctrl + C to wake them up.

If you have any other information about your environment that could be causing the minutes delay please let me know and I'll try to reproduce the conditions.

[Tiberriver256]

5-6 seconds would be awesome. Both of us however are experiencing minutes.

[Olian04]

@MarcoScabbiolo don't know if it makes any difference but I'm installing yo locally and running it via a globally installed npx

[Tiberriver256]

Seems like most of the time for me is spent here: https://github.com/yeoman/environment/blob/8bfa1ed05eb74dc03c89fd373cfa2b941259f121/lib/resolver.js#L75-L78

[MarcoScabbiolo]

Ran yo with npx installing it locally and had the same delay, 5-6 seconds. Can you share your package.json file to see if the ammount of dependencies is having an impact on the time it takes to resolve all modules to see which of them are candidates for a generator?

[MarcoScabbiolo]

By the way I'm running these on a pretty beafy PC, 18 GB of RAM and an i7-7700 with two SSDs , but I don't think this accounts for the huge difference in the delay.

[Olian04]

@MarcoScabbiolo Just in case it matters; I'm on an i7-6820HQ with 16GB RAM and an ok SSD.

As suggested in #589 i tried using #590/yo@2.0.4 but its the same result. Slow start, but once the first log line is printed it runs smooth and quick.

[MarcoScabbiolo]

@SBoudrias Looking at what @Tiberriver256 said it doesnt look like global-tunnel-ng is the reason for these delays.

If Im able to reproduce the issue ill try to debug it.

[Tiberriver256]

I completely cleaned out my global node_modules folder and it seems to be back down to 5-6 seconds.

@Olian04 how many global modules do you have out of curiousity?

(dir $Env:APPDATA\npm\node_modules\).count

[Olian04]

@Tiberriver256 "one". npx and its deps (since I'm using LTS I have to install in on my own). I install everything locally to ensure that I always know what versions I'm running. That includes yo.

[MarcoScabbiolo]

Many places are searched while trying to resolve the list of available generators, for example if I run npx yo on D:\Tmp\node, all these directories will be recursively searched:

C:\Users\username\AppData\Roaming\nvm\v8.11.3\node_modules
d:\Tmp\yo
C:\Users\username\AppData\Local\Yarn\Data\global\node_modules
C:\Users\username\AppData\Local\Yarn\Data\link
C:\Users\username\AppData\Roaming\npm\lib\node_modules
d:\node_modules
d:\Tmp\node_modules
d:\Tmp\yo\node_modules
d:\Tmp\yo\node\node_modules

[wsrast]

I'll throw my hat in the ring here as well, coming from another issue I posted and redirected here by @SBoudrias .

I'm now experiencing this on two separate Windows machines. Things I've tried:

1. Uninstalled NVM and all versions of Node it contained.
2. Reinstalled Node LTS 8.11.3
3. Used both private and public NPM registries.
4. Installed Yeoman 2.0.4 with Yarn instead of NPM (this bypasses another issue I've created with any Yeoman version >2.0.2 on an NPM install behind a corporate proxy).
5. Ran powershell command suggested by @Tiberriver256 . Delay was still in evidence (~3-5 minutes). Output was:

   PS C:\Users\rastwe> $Env:DEBUG = "yeoman:*"
   PS C:\Users\myuser> yo
   ? 'Allo rastwe! What would you like to do? Get me out of here!
   Terminate batch job (Y/N)? y
   PS C:\Users\myuser>

I am behind a corporate proxy, but the speed of the proxy doesn't seem to be an issue in any other application.

Can someone give some insight in how you're producing your time-require logs? I'd be interested to try that out, but don't see anything obvious on how to use that from the command line in the official documentation.

[Olian04]

@wsrast take a look at #589

[wsrast]

@Olian04 Thanks, these issues do seem to all revolve around proxy settings. I've subscribed to #589 and will see if I can add anything to the conversation as I continue testing.

[MarcoScabbiolo]

If you're having this issue running yo behind a proxy please try #597 setting the DEBUG environment variable to global-tunnel and share the output.

[wsrast]

A simple set DEBUG=global-tunnel shows no output for me on a yo --generators command on Windows.

[SBoudrias]

@wsrast did you try to update yo? (uninstall/reinstall?)

[wsrast]

Yes, I've tried that several times while working with the original issue.

[MarcoScabbiolo]

@wsrast Last version of yo that includes the additional logging isn't published to npm yet, that's why you don't get the debug info.

[SBoudrias]

2.0.5 is out. My bad, I didn't know npm respect package-lock.json for global install.

[wsrast]

Here's my update using 2.0.5. I've tried in turn using proxy, http-proxy, and https-proxy settings in my global .npmrc file, and all are functional, but they all show the ~3-5 minute delay. Here's the debugging information from the global-tunnel output. The long pause occurs after the second "Creating proxying agent" statement, and before "Available Generators:". It's worth noting that all the lines before the pause show up in just a few seconds.

set DEBUG=global-tunnel && yo --generators DEBUG global-tunnel: Found proxy in npm config proxy DEBUG global-tunnel: Proxy configuration to be used is { "protocol": "http:", "host": "web.company.net", "port": 8080, "proxyAuth": "user:pw" } DEBUG global-tunnel: Creating proxying agent DEBUG global-tunnel: Creating proxying agent Available Generators: @company/package

[wsrast]

I've just reverted to version 1.8.5 and performance has improved to less than 1 second for the same yo --generators command. At this point, I'll be recommending to all the business units I work with to avoid v2.0 due to the large performance hit.

[MarcoScabbiolo]

Definitively the delays are not caused by the proxy wrapper introduced, yo --generators does not do any http request, and if it did, you would see it logged as you can see when you run yo doctor

yo doctor
DEBUG global-tunnel: Found proxy in environment variable http_proxy
DEBUG global-tunnel: Proxy configuration to be used is {
  "protocol": "http:",
  "host": "localhost",
  "port": 8080,
  "proxyAuth": null
}
DEBUG global-tunnel: Creating proxying agent
DEBUG global-tunnel: Creating proxying agent

Yeoman Doctor
Running sanity checks on your system

✔ Global configuration file is valid
✔ NODE_PATH matches the npm root
✔ Node.js version
DEBUG global-tunnel: Requesting to https://registry.npmjs.org:443
✔ No .bowerrc file in home directory
✔ No .yo-rc.json file in home directory
✔ npm version
✔ yo version

Everything looks all right!

It looks like the discovery of generators is taking too long, probably because its looking in too many directories recursively.

[wsrast]

@MarcoScabbiolo This occurs with calls to individual generators as well, not just "yo" or "yo --generators".

[MarcoScabbiolo]

The cli looks up for available generators running env.lookup on the init function that is always invoked. It doesn't matter what arguments you pass to yo the generators are always discovered and this is what is causing the delays.

[SBoudrias]

Anyone running into the issue could run a profiling with chrome dev tool remote debugging? This would be extremely useful and likely to point to the core issues.

It could just be a library we bumped that has a performance regression in an hot path of yo.

[martaver]

Just a thought, I'm using yeoman on Win10 with WSL, and its slow too. Are you by any chance using yarn link, or similar thing, to run a local generator too?

[SBoudrias]

Ran a trace with clinic:

clinic flame -- nodewhich yo--generators (for windows users you won't have which available, but where is roughly equivalent on powershell)

Looks like most of the time is spent inside @mrmlnc/readdir-enhanced which was updated when globby was updated to use fast-glob.

You can copy-paste this HTML in jsbin to explore the graph https://gist.github.com/SBoudrias/3004406099a97f05478305311e5dc3a3

[andersaloof]

I'm on Win 7 and I've been quite annoyed at the slowness in running generators too, so I did some console.log debugging to try to figure out whats happening.

I managed to cut almost 50% off time used before a prompt in an internal generator appears, by doing some small modifications to resolver.js, PR is here https://github.com/yeoman/environment/pull/105.

However, the biggest problem is definately somewhere in fast-glob or readdir-enhanced, as stated by @SBoudrias above, to me it seems to be using an unreasonable long time searching for generators with the @*/generators-* selector.

[andersaloof]

After some more debugging, it looks to me like the underlying issue is with https://github.com/BigstickCarpet/readdir-enhanced/blob/master/lib/directory-reader.js, it seems to recurse folders which does not match the specified @*/generator-* pattern, making it traverse entire node_modules structures, when it only should be recursing into folders matching the pattern...

[MarcoScabbiolo]

globby was updated to use fast-glob which depends on micromatch that has a performance issue.

yeoman-environment updated to version 8> of globby introducing the above issue, which was then introduced into yo when yeoman-environment was bumped

[andersaloof]

I've cut down time to prompt further, by removing lookups in non-namespace folders when searching for generators in namespace folders. A PR is here https://github.com/yeoman/environment/pull/106.

In my current project, I've gone from around 20s until prompt from a generator appears, to 4s with the two submitted PRs.