Open jstevans opened 5 years ago
Running this on itself (see the benchmark-processFile
branch), I see:
STARTED
DONE IN 86788.95489120483ms
TOP ENTRIES:
processFile{node_modules/lodash/lodash.js} - 158.653432
processFile{node_modules/neo-async/async.js} - 101.587037
processFile{node_modules/source-map-resolve/source-map-resolve.js} - 640.555826
processFile{node_modules/ajv/dist/ajv.bundle.js} - 194.803764
processFile{node_modules/ajv/dist/ajv.min.js} - 109.212528
processFile{node_modules/ajv/lib/cache.js} - 364.88105
processFile{node_modules/ajv/lib/data.js} - 101.987961
processFile{node_modules/expect/build-es5/index.js} - 622.996264
processFile{node_modules/handlebars/dist/handlebars.amd.js} - 215.928614
processFile{node_modules/handlebars/dist/handlebars.amd.min.js} - 194.337775
processFile{node_modules/js-yaml/dist/js-yaml.js} - 116.438963
processFile{node_modules/lodash/fp/result.js} - 1527.005315
processFile{node_modules/lodash/fp/unzipWith.js} - 571.401291
processFile{node_modules/requirejs/bin/r.js} - 684.083539
processFile{node_modules/typescript/lib/lib.dom.d.ts} - 274.261523
processFile{node_modules/typescript/lib/tsc.js} - 4546.898113
processFile{node_modules/typescript/lib/tsserver.js} - 11411.919684
processFile{node_modules/typescript/lib/tsserverlibrary.js} - 11299.118504
processFile{node_modules/typescript/lib/typescript.js} - 9512.589519
processFile{node_modules/typescript/lib/typescriptServices.js} - 9707.257702
processFile{node_modules/typescript/lib/typingsInstaller.js} - 6333.728866
processFile{node_modules/uglify-js/lib/compress.js} - 123.569683
processFile{node_modules/@babel/parser/lib/index.js} - 175.728574
processFile{node_modules/@babel/types/lib/index.js} - 202.899436
processFile{node_modules/handlebars/dist/amd/precompiler.js} - 321.986059
processFile{node_modules/jsdom/lib/jsdom/browser/documentfeatures.js} - 159.730241
processFile{node_modules/jsdom/lib/jsdom/living/named-properties-window.js} - 505.641625
processFile{node_modules/jsdom/lib/jsdom/living/node-document-position.js} - 103.477281
processFile{node_modules/jsdom/lib/jsdom/living/post-message.js} - 500.039741
processFile{node_modules/jsdom/lib/jsdom/living/xhr-sync-worker.js} - 129.355988
processFile{node_modules/jsdom/lib/jsdom/living/xmlhttprequest-symbols.js} - 207.125743
processFile{node_modules/mkdirp/node_modules/minimist/example/parse.js} - 121.234851
processFile{node_modules/mkdirp/node_modules/minimist/test/dash.js} - 206.668863
processFile{node_modules/handlebars/dist/amd/handlebars/compiler/code-gen.js} - 172.938695
total of top entries: 61620.04405
total processFile{node_modules}: 83571.53660699965
total processFile{repo}: 277.72363500000006
Looks like processFile
took up ~96% of time, and ~99% of that was spent in node_modules
.
Currently, we don't give special treatment to
node_modules
-- if there's a filenode_modules/myPkg/bar.js
, then it gets hashed and dependency-checked the same as any other file in the repo; and if if a repo's filesrc/foo.js
depends onnode_modules/pkg/bar.js
, then that dependency is presented in the graph in the same way as a dependency on another repo file.In most projects,
node_modules
seems to be both much larger and slower to change than files in the repo itself. This being the case, it would be good to have a configfuzzyNodeModules
that handlesnode_modules
at the package level instead of the file level. Using the same example withsrc/foo.js
andnode_modules/myPkg/bar.js
:src/foo.js
will be treated as depending on thenode_modules/myPkg
package (rather than onnode_modules/myPkg/bar.js
specifically)node_modules/myPkg
node in the dependency graph that contains the recursive file-hash of the whole package (rather than each file innode_modules/myPkg
appearing in the dependency graph)