Reduce installed package size

Latest master (b79fdfd8b0e5632eae338b563af1c3be29fecbca), when installed with npm@3.5.2, is 68 MB. Most of that is Babel stuff.

We should look into ways we can reduce the size.

(Output from DaisyDisk)

Turns out every single Babel package includes a dependency on babel-runtime and for some reason it's not deduped by npm@3, not even with npm dedupe:

~/dev/ava/node_modules master*
❯ find . -type d -name "babel-runtime" -print 
./babel-code-frame/node_modules/babel-runtime
./babel-core/node_modules/babel-runtime
./babel-generator/node_modules/babel-runtime
./babel-helper-builder-binary-assignment-operator-visitor/node_modules/babel-runtime
./babel-helper-call-delegate/node_modules/babel-runtime
./babel-helper-define-map/node_modules/babel-runtime
./babel-helper-explode-assignable-expression/node_modules/babel-runtime
./babel-helper-function-name/node_modules/babel-runtime
./babel-helper-get-function-arity/node_modules/babel-runtime
./babel-helper-hoist-variables/node_modules/babel-runtime
./babel-helper-optimise-call-expression/node_modules/babel-runtime
./babel-helper-regex/node_modules/babel-runtime
./babel-helper-remap-async-to-generator/node_modules/babel-runtime
./babel-helper-replace-supers/node_modules/babel-runtime
./babel-helpers/node_modules/babel-runtime
./babel-messages/node_modules/babel-runtime
./babel-plugin-check-es2015-constants/node_modules/babel-runtime
./babel-plugin-syntax-async-functions/node_modules/babel-runtime
./babel-plugin-syntax-exponentiation-operator/node_modules/babel-runtime
./babel-plugin-syntax-object-rest-spread/node_modules/babel-runtime
./babel-plugin-transform-async-to-generator/node_modules/babel-runtime
./babel-plugin-transform-es2015-arrow-functions/node_modules/babel-runtime
./babel-plugin-transform-es2015-block-scoped-functions/node_modules/babel-runtime
./babel-plugin-transform-es2015-block-scoping/node_modules/babel-runtime
./babel-plugin-transform-es2015-classes/node_modules/babel-runtime
./babel-plugin-transform-es2015-computed-properties/node_modules/babel-runtime
./babel-plugin-transform-es2015-destructuring/node_modules/babel-runtime
./babel-plugin-transform-es2015-for-of/node_modules/babel-runtime
./babel-plugin-transform-es2015-function-name/node_modules/babel-runtime
./babel-plugin-transform-es2015-literals/node_modules/babel-runtime
./babel-plugin-transform-es2015-modules-commonjs/node_modules/babel-runtime
./babel-plugin-transform-es2015-object-super/node_modules/babel-runtime
./babel-plugin-transform-es2015-parameters/node_modules/babel-runtime
./babel-plugin-transform-es2015-shorthand-properties/node_modules/babel-runtime
./babel-plugin-transform-es2015-spread/node_modules/babel-runtime
./babel-plugin-transform-es2015-sticky-regex/node_modules/babel-runtime
./babel-plugin-transform-es2015-template-literals/node_modules/babel-runtime
./babel-plugin-transform-es2015-typeof-symbol/node_modules/babel-runtime
./babel-plugin-transform-es2015-unicode-regex/node_modules/babel-runtime
./babel-plugin-transform-exponentiation-operator/node_modules/babel-runtime
./babel-plugin-transform-object-rest-spread/node_modules/babel-runtime
./babel-plugin-transform-regenerator/node_modules/babel-runtime
./babel-plugin-transform-runtime/node_modules/babel-runtime
./babel-plugin-transform-strict-mode/node_modules/babel-runtime
./babel-register/node_modules/babel-runtime
./babel-runtime
./babel-template/node_modules/babel-runtime
./babel-traverse/node_modules/babel-runtime
./babel-types/node_modules/babel-runtime
./babylon/node_modules/babel-runtime

@sindresorhus: That appears to be a bug on your machine. On mine it works fine: Screenshot ls spits out the size kind of weird (in raw bytes, floored), it's actually 33.2MB: npm ls:

/Users/Ari/Developer/.tmp
└─┬ ava@0.8.0
  ├── arr-flatten@1.0.1
  ├── arrify@1.0.1
  ├─┬ ava-init@0.1.3
  │ ├── arr-exclude@1.0.0
  │ ├── pify@2.3.0
  │ ├─┬ pinkie-promise@2.0.0
  │ │ └── pinkie@2.0.1
  │ ├─┬ read-pkg-up@1.0.1
  │ │ ├─┬ find-up@1.1.0
  │ │ │ └── path-exists@2.1.0
  │ │ └─┬ read-pkg@1.1.0
  │ │   ├─┬ load-json-file@1.1.0
  │ │   │ ├─┬ parse-json@2.2.0
  │ │   │ │ └─┬ error-ex@1.3.0
  │ │   │ │   └── is-arrayish@0.2.1
  │ │   │ └─┬ strip-bom@2.0.0
  │ │   │   └── is-utf8@0.2.1
  │ │   └── path-type@1.1.0
  │ ├── the-argv@1.0.0
  │ └─┬ write-pkg@1.0.0
  │   └─┬ write-json-file@1.2.0
  │     └─┬ sort-keys@1.1.1
  │       └── is-plain-obj@1.1.0
  ├─┬ babel-core@5.8.34
  │ ├── babel-plugin-constant-folding@1.0.1
  │ ├── babel-plugin-dead-code-elimination@1.0.2
  │ ├── babel-plugin-eval@1.0.1
  │ ├── babel-plugin-inline-environment-variables@1.0.1
  │ ├── babel-plugin-jscript@1.0.4
  │ ├── babel-plugin-member-expression-literals@1.0.1
  │ ├── babel-plugin-property-literals@1.0.1
  │ ├── babel-plugin-proto-to-assign@1.0.4
  │ ├── babel-plugin-react-constant-elements@1.0.3
  │ ├── babel-plugin-react-display-name@1.0.3
  │ ├── babel-plugin-remove-console@1.0.1
  │ ├── babel-plugin-remove-debugger@1.0.1
  │ ├── babel-plugin-runtime@1.0.7
  │ ├─┬ babel-plugin-undeclared-variables-check@1.0.2
  │ │ └── leven@1.0.2
  │ ├── babel-plugin-undefined-to-void@1.1.6
  │ ├── babylon@5.8.34
  │ ├── bluebird@2.10.2
  │ ├── convert-source-map@1.1.2
  │ ├── core-js@1.2.6
  │ ├─┬ detect-indent@3.0.1
  │ │ ├── get-stdin@4.0.1
  │ │ └── minimist@1.2.0
  │ ├── esutils@2.0.2
  │ ├── fs-readdir-recursive@0.1.2
  │ ├── globals@6.4.1
  │ ├─┬ home-or-tmp@1.0.0
  │ │ ├── os-tmpdir@1.0.1
  │ │ └── user-home@1.1.1
  │ ├── is-integer@1.0.6
  │ ├── js-tokens@1.0.1
  │ ├── json5@0.4.0
  │ ├─┬ line-numbers@0.2.0
  │ │ └── left-pad@0.0.3
  │ ├── lodash@3.10.1
  │ ├─┬ minimatch@2.0.10
  │ │ └─┬ brace-expansion@1.1.2
  │ │   ├── balanced-match@0.3.0
  │ │   └── concat-map@0.0.1
  │ ├─┬ output-file-sync@1.1.1
  │ │ └─┬ mkdirp@0.5.1
  │ │   └── minimist@0.0.8
  │ ├── path-exists@1.0.0
  │ ├── path-is-absolute@1.0.0
  │ ├── private@0.1.6
  │ ├─┬ regenerator@0.8.40
  │ │ ├─┬ commoner@0.10.4
  │ │ │ ├─┬ commander@2.9.0
  │ │ │ │ └── graceful-readlink@1.0.1
  │ │ │ ├─┬ detective@4.3.1
  │ │ │ │ ├── acorn@1.2.2
  │ │ │ │ └── defined@1.0.0
  │ │ │ ├── glob@5.0.15
  │ │ │ ├── iconv-lite@0.4.13
  │ │ │ └── q@1.4.1
  │ │ ├─┬ defs@1.1.1
  │ │ │ ├─┬ alter@0.2.0
  │ │ │ │ └── stable@0.1.5
  │ │ │ ├── ast-traverse@0.1.1
  │ │ │ ├── breakable@1.0.0
  │ │ │ ├── simple-fmt@0.1.0
  │ │ │ ├── simple-is@0.2.0
  │ │ │ ├── stringmap@0.2.2
  │ │ │ ├── stringset@0.2.1
  │ │ │ ├── tryor@0.1.2
  │ │ │ └─┬ yargs@3.27.0
  │ │ │   ├── camelcase@1.2.1
  │ │ │   ├─┬ cliui@2.1.0
  │ │ │   │ ├─┬ center-align@0.1.2
  │ │ │   │ │ ├─┬ align-text@0.1.3
  │ │ │   │ │ │ ├─┬ kind-of@2.0.1
  │ │ │   │ │ │ │ └── is-buffer@1.1.0
  │ │ │   │ │ │ └── repeat-string@1.5.2
  │ │ │   │ │ └── lazy-cache@0.2.7
  │ │ │   │ └── right-align@0.1.3
  │ │ │   ├── decamelize@1.1.2
  │ │ │   ├─┬ os-locale@1.4.0
  │ │ │   │ └─┬ lcid@1.0.0
  │ │ │   │   └── invert-kv@1.0.0
  │ │ │   ├── window-size@0.1.4
  │ │ │   └── y18n@3.2.0
  │ │ ├── esprima-fb@15001.1001.0-dev-harmony-fb
  │ │ ├─┬ recast@0.10.33
  │ │ │ └── ast-types@0.8.12
  │ │ └── through@2.3.8
  │ ├─┬ regexpu@1.3.0
  │ │ ├── esprima@2.7.1
  │ │ ├── regenerate@1.2.1
  │ │ ├── regjsgen@0.2.0
  │ │ └─┬ regjsparser@0.1.5
  │ │   └── jsesc@0.5.0
  │ ├── repeating@1.1.3
  │ ├── resolve@1.1.6
  │ ├── shebang-regex@1.0.0
  │ ├── slash@1.0.0
  │ ├── source-map@0.5.3
  │ ├─┬ source-map-support@0.2.10
  │ │ └── source-map@0.1.32
  │ ├── to-fast-properties@1.0.1
  │ ├── trim-right@1.0.1
  │ └── try-resolve@1.0.1
  ├─┬ babel-plugin-espower@1.1.0
  │ ├── array-find@1.0.0
  │ ├─┬ escallmatch@1.4.2
  │ │ ├── deep-equal@1.0.1
  │ │ ├── esprima@2.7.1
  │ │ ├─┬ espurify@1.5.0
  │ │ │ └── isarray@1.0.0
  │ │ └── indexof@0.0.1
  │ ├─┬ espower@1.2.1
  │ │ ├─┬ escodegen@1.7.1
  │ │ │ ├── esprima@1.2.5
  │ │ │ ├── estraverse@1.9.3
  │ │ │ ├─┬ optionator@0.5.0
  │ │ │ │ ├── deep-is@0.1.3
  │ │ │ │ ├── fast-levenshtein@1.0.7
  │ │ │ │ ├── levn@0.2.5
  │ │ │ │ ├── prelude-ls@1.1.2
  │ │ │ │ ├── type-check@0.3.1
  │ │ │ │ └── wordwrap@0.0.2
  │ │ │ └── source-map@0.2.0
  │ │ ├── is-url@1.2.1
  │ │ └── isarray@0.0.1
  │ └── xtend@4.0.1
  ├── babel-runtime@5.8.34
  ├── bluebird@3.1.1
  ├─┬ chalk@1.1.1
  │ ├── ansi-styles@2.1.0
  │ ├── escape-string-regexp@1.0.4
  │ ├─┬ has-ansi@2.0.0
  │ │ └── ansi-regex@2.0.0
  │ ├── strip-ansi@3.0.0
  │ └── supports-color@2.0.0
  ├─┬ co-with-promise@4.6.0
  │ └─┬ pinkie-promise@1.0.0
  │   └── pinkie@1.0.0
  ├─┬ core-assert@0.1.3
  │ └── buf-compare@1.0.0
  ├─┬ debug@2.2.0
  │ └── ms@0.7.1
  ├── deeper@2.1.0
  ├─┬ empower@1.1.0
  │ ├── array-filter@1.0.0
  │ ├── array-foreach@1.0.1
  │ ├── array-map@0.0.0
  │ ├── array-some@1.0.0
  │ ├─┬ define-properties@1.1.2
  │ │ └── foreach@2.0.5
  │ └─┬ object-create@0.1.0
  │   └─┬ object-define-property@0.1.0
  │     ├── function-bind@0.1.0
  │     └── has@0.0.1
  ├─┬ empower-core@0.2.0
  │ └── call-signature@0.0.2
  ├── figures@1.4.0
  ├── fn-name@2.0.1
  ├─┬ globby@4.0.0
  │ ├─┬ array-union@1.0.1
  │ │ └── array-uniq@1.0.2
  │ └─┬ glob@6.0.2
  │   ├─┬ inflight@1.0.4
  │   │ └── wrappy@1.0.1
  │   ├── inherits@2.0.1
  │   └── once@1.3.3
  ├── has-generator@1.0.0
  ├── is-generator-fn@1.0.0
  ├─┬ is-observable@0.1.0
  │ └── symbol-observable@0.1.0
  ├── is-promise@2.1.0
  ├─┬ loud-rejection@1.2.0
  │ └── signal-exit@2.1.2
  ├── max-timeout@1.0.0
  ├─┬ meow@3.6.0
  │ ├─┬ camelcase-keys@2.0.0
  │ │ ├── camelcase@2.0.1
  │ │ └── map-obj@1.0.1
  │ ├── minimist@1.2.0
  │ ├─┬ normalize-package-data@2.3.5
  │ │ ├── hosted-git-info@2.1.4
  │ │ ├─┬ is-builtin-module@1.0.0
  │ │ │ └── builtin-modules@1.1.0
  │ │ ├── semver@5.1.0
  │ │ └─┬ validate-npm-package-license@3.0.1
  │ │   ├─┬ spdx-correct@1.0.2
  │ │   │ └── spdx-license-ids@1.1.0
  │ │   └─┬ spdx-expression-parse@1.0.2
  │ │     └── spdx-exceptions@1.0.4
  │ ├─┬ redent@1.0.0
  │ │ ├─┬ indent-string@2.1.0
  │ │ │ └── repeating@2.0.0
  │ │ └── strip-indent@1.0.1
  │ └── trim-newlines@1.0.0
  ├── object-assign@4.0.1
  ├── observable-to-promise@0.1.0
  ├─┬ plur@2.1.2
  │ └── irregular-plurals@1.1.0
  ├─┬ power-assert-formatter@1.3.2
  │ ├── acorn@2.6.4
  │ ├── acorn-es7-plugin@1.0.11
  │ ├── array-reduce@0.0.0
  │ ├── eastasianwidth@0.1.1
  │ ├── estraverse@4.1.1
  │ ├── googlediff@0.1.0
  │ ├── object-keys@1.0.9
  │ ├─┬ stringifier@1.2.1
  │ │ ├── array-reduce-right@1.0.0
  │ │ └── traverse@0.6.6
  │ └── type-name@1.1.0
  ├── power-assert-renderers@0.1.0
  ├─┬ pretty-ms@2.1.0
  │ ├─┬ is-finite@1.0.1
  │ │ └── number-is-nan@1.0.0
  │ ├── parse-ms@1.0.0
  │ └── plur@1.0.0
  ├── require-from-string@1.1.0
  ├─┬ resolve-cwd@1.0.0
  │ └── resolve-from@2.0.0
  ├── serialize-error@1.1.0
  ├── set-immediate-shim@1.0.1
  ├─┬ source-map-support@0.4.0
  │ └─┬ source-map@0.1.32
  │   └── amdefine@1.0.0
  ├─┬ squeak@1.3.0
  │ ├── console-stream@0.1.1
  │ └─┬ lpad-align@1.1.0
  │   ├── longest@1.0.1
  │   └── lpad@2.0.1
  ├─┬ time-require@0.1.2
  │ ├─┬ chalk@0.4.0
  │ │ ├── ansi-styles@1.0.0
  │ │ ├── has-color@0.1.7
  │ │ └── strip-ansi@0.1.1
  │ ├── date-time@0.1.1
  │ ├─┬ pretty-ms@0.2.2
  │ │ └── parse-ms@0.1.2
  │ └── text-table@0.2.0
  └─┬ update-notifier@0.5.0
    ├─┬ configstore@1.4.0
    │ ├── graceful-fs@4.1.2
    │ ├─┬ osenv@0.1.3
    │ │ └── os-homedir@1.0.1
    │ ├── uuid@2.0.1
    │ ├─┬ write-file-atomic@1.1.4
    │ │ ├── imurmurhash@0.1.4
    │ │ └── slide@1.1.6
    │ └── xdg-basedir@2.0.0
    ├── is-npm@1.0.0
    ├─┬ latest-version@1.0.1
    │ └─┬ package-json@1.2.0
    │   ├─┬ got@3.3.1
    │   │ ├─┬ duplexify@3.4.2
    │   │ │ ├── end-of-stream@1.0.0
    │   │ │ └─┬ readable-stream@2.0.5
    │   │ │   ├── core-util-is@1.0.2
    │   │ │   ├── isarray@0.0.1
    │   │ │   ├── process-nextick-args@1.0.6
    │   │ │   ├── string_decoder@0.10.31
    │   │ │   └── util-deprecate@1.0.2
    │   │ ├── infinity-agent@2.0.3
    │   │ ├── is-redirect@1.0.0
    │   │ ├── is-stream@1.0.1
    │   │ ├── lowercase-keys@1.0.0
    │   │ ├── nested-error-stacks@1.0.2
    │   │ ├── object-assign@3.0.0
    │   │ ├── prepend-http@1.0.3
    │   │ ├─┬ read-all-stream@3.0.1
    │   │ │ └─┬ pinkie-promise@1.0.0
    │   │ │   └── pinkie@1.0.0
    │   │ └── timed-out@2.0.0
    │   └─┬ registry-url@3.0.3
    │     └─┬ rc@1.1.5
    │       ├── deep-extend@0.4.0
    │       ├── ini@1.3.4
    │       ├── minimist@1.2.0
    │       └── strip-json-comments@1.0.4
    ├── semver-diff@2.1.0
    └── string-length@1.0.1

So I'm really not sure why you're seeing that. Maybe file a bug report with npm?

Actually, it is only ~16.5 MB, @ariporad's number includes dev-dependencies. You need to delete node_modules and run

$ npm install --production

If you create a new project and just do this:

$ npm init
# default for every prompt
$ npm i babel-core babel-preset-es2015 babel-preset-stage-2

I get a relatively svelte 6.8 MB.

Adding babel-plugin-espower brings me to 9.8 MB. That seems like a pretty significant jump. @twada - Any ideas how we can reduce this?

Adding babel-runtime@6.X.X does bump me up to 14 MB, but that is due to massive duplication of babel-runtime@5.X.X. It gets copied into the node_modules folder of every babel-plugin-transform-*. This is expected, since current babel plugins are written in ES6, compiled with babel@5, and maintain a dev-dependency on babel-runtime@5. This really does not hurt your install times though, since all those duplicates are definitely coming from the cache (babel-runtime is not re-downloaded 20 times).

2.5 MB for the remaining 30 some dependencies we have seems pretty reasonable.

@jamestalmage: that's strange, I did go above and beyond my usual intelligence level and create a new project to install AVA, but I'll try again when I'm home.

What are we hoping to achieve here? With changing dependencies this is a never ending struggle.

That said, looks like all Babel dependencies require babel-runtime@^5, whereas we explicitly install ^6. There's FIFTY copies of babel-runtime@5.8.35.

Presumably we could downgrade to ^5 for the time being and we'd save a lot space. In my testing that takes us from 77MB to 29MB.

Of course there's other dependencies doing weird stuff, like fsevents which pulls in 7MB just for node-pre-gyp.

To measure I did a fresh install from NPM (in an empty directory) and ran du -k -d 1 node_modules | sort -n -r. Also check out snyk-resolve which shows the actual dependencies, not deduped.

@novemberborn There are some things that can be done even if the worst offender, Babel and fsevents, are out of our control. Many packages include junk like tests or examples directory, etc. I did a PR marathon a few years ago for Yeoman on 100+ different packages adding a "files" property to their package.json to reduce the total disk size of the dependency tree. That helped a lot. Pretty much have this still open for when I have time to do the same here.

Pretty much have this still open for when I have time to do the same here.

Sure, fair enough. Somebody should write a tool for that :stuck_out_tongue_winking_eye:

Also thinking about it more I'm doubtful we can change our babel-runtime version. Presumably the runtime transformer plugin builds against v6, not v5.

Also thinking about it more I'm doubtful we can change our babel-runtime version. Presumably the runtime transformer plugin builds against v6, not v5.

Actually if we have a dependency which merely reexports babel-runtime it would allow npm to dedupe babel-runtime@5 instead.

Actually if we have a dependency which merely reexports babel-runtime it would allow npm to dedupe babel-runtime@5 instead.

I thought about that. Actually, I think it would be better if Babel did that (all the Babel 6 plugins should depend on a differently named module that exports babel-runtime@5). That way everyone benefits.

Actually, I think it would be better if Babel did that (all the Babel 6 plugins should depend on a differently named module that exports babel-runtime@5). That way everyone benefits.

:+1:

Is detecting Node v6 possible or an option? Would we still want babel to be part of ava when the environment is ES2015-compliant?

@jokeyrhyme AVA still injects features that have not yet landed in ES2015 or ES2016, such as async/await. Additionally, it uses Babel for internal transforms (enhancing assertions with power-assert for instance).

I don't know if it's possible to correctly detect the Node version, but that'd be at runtime, not at install time. The team could remove the ES2015 preset when all supported Node versions support it, but that could take a while.

@jfmengels we might be able to install babel in a pre/post-install npm hook, and only install exactly the transforms needed to get us from the current version of node to stage-2 or whatever our threshold is. That would mean fewer dependencies for newer Nodes?

@jokeyrhyme That's unfortunately not an option for so many reasons. Some of the best features of AVA require Babel, so I don't see why anyone would want to drop it for some minor space savings. The biggest Babel size issue is about to get fixed anyways https://github.com/babel/babel/pull/3438.

If you want to help out with this, I'd suggest you find packages in the dependency tree that includes junk like test/examples/etc folders and ask them (or better yet do a PR) to use the files property in package.json.

Here's a list of packages we depend on that doesn't have a files entry or .npmignore. Please help us reduce the size of AVA by submitting PRs to those packages adding a files property in package.json.

Tip: $ npm i -g npm-home and then just write nh package-name to go to a specific package.

[x] acorn-jsx@3.0.1 - master has .npmignore
[x] amdefine@1.0.0 - master has .npmignore
[x] argparse@0.1.16 - master has files entry in package.json
[ ] array-map@0.0.0 - https://github.com/substack/array-map/pull/6
[ ] array-reduce@0.0.0 - https://github.com/substack/array-reduce/pull/4
[x] assert-plus@0.1.5 - master has .npmignore
[x] assert-plus@0.2.0
[x] assert-plus@1.0.0
[x] async@0.2.10 - master has .npmignore
[x] async@0.9.2
[x] aws-sign2@0.5.0 - no extra files
[x] aws-sign2@0.6.0
[ ] babel-core@6.8.0
[x] block-stream@0.0.8 - master has .npmignore
[ ] caseless@0.11.0 - https://github.com/request/caseless/pull/26
[ ] caseless@0.6.0
[ ] combined-stream@0.0.7
[ ] combined-stream@1.0.5
[ ] commondir@1.0.1 - https://github.com/substack/node-commondir/pull/6
[ ] concat-map@0.0.1 - https://github.com/substack/node-concat-map/pull/5
[x] core-js@1.2.6 - has .npmignore
[x] core-js@2.3.0
[ ] core-util-is@1.0.2 - https://github.com/isaacs/core-util-is/pull/13
[x] dashdash@1.13.0 - master has .npmignore
[ ] dashdash@1.13.1
[ ] deep-equal@0.1.2 https://github.com/substack/node-deep-equal/pull/39
[ ] deep-equal@1.0.1
[ ] defined@0.0.0 - https://github.com/substack/defined/pull/3
[ ] eastasianwidth@0.1.1 - https://github.com/komagata/eastasianwidth/pull/3
[x] escope@3.6.0 - master has .npmignore
[x] esrecurse@4.1.0 - master has .npmignore
[x] estraverse@1.9.3 - master has .npmignore
[ ] estraverse@4.1.1
[ ] estraverse@4.2.0
[ ] events-to-array@1.0.2 - https://github.com/isaacs/events-to-array/pull/1
[x] extsprintf@1.0.2 - master has .npmignore
[ ] forever-agent@0.5.2 - https://github.com/request/forever-agent/pull/35
[ ] forever-agent@0.6.1
[x] form-data@0.1.4 - master has .npmignore
[ ] form-data@1.0.0-rc4
[ ] formatio@1.1.1 - https://github.com/busterjs/formatio/pull/8
[ ] googlediff@0.1.0 - https://github.com/shimondoodkin/googlediff/pull/6
[ ] inflight@1.0.4 - https://github.com/npm/inflight/pull/2
[ ] inherits@2.0.1 - https://github.com/isaacs/inherits/pull/21
[ ] is-buffer@1.1.3 - Author refuses to fix: https://github.com/feross/is-buffer/pull/12
[ ] is-typedarray@1.0.0 - https://github.com/hughsk/is-typedarray/pull/2
[x] isarray@0.0.1 - https://github.com/juliangruber/isarray/pull/9
[x] js-yaml@3.0.1 - has files entry in package.json
[ ] json-schema@0.2.2 - https://github.com/kriszyp/json-schema/pull/68
[ ] jsonify@0.0.0 - https://github.com/substack/jsonify/pull/6
[ ] jsonpointer@2.0.0 - https://github.com/janl/node-jsonpointer/pull/26
[x] jsprim@1.2.2 - master has .npmignore
[x] mime@1.2.11 - master has .npmignore
[ ] minimist@0.0.8 - PR already filed
[ ] minimist@1.2.0
[ ] mkdirp@0.5.1 - https://github.com/substack/node-mkdirp/pull/104
[ ] mute-stream@0.0.5 - https://github.com/npm/mute-stream/pull/8
[ ] mute-stream@0.0.6
[x] nan@2.3.3 - master has .npmignore
[x] node-pre-gyp@0.6.25 - master has .npmignore
[ ] npmlog@2.0.3 - https://github.com/npm/npmlog/pull/32
[x] oauth-sign@0.4.0 - https://github.com/request/oauth-sign/pull/21
[ ] oauth-sign@0.8.1
[ ] optimist@0.6.1 - https://github.com/substack/node-optimist/pull/142
[x] path-is-inside@1.0.1 - has .npmignore
[ ] private@0.1.6 - https://github.com/benjamn/private/pull/11
[x] process-nextick-args@1.0.6 - MERGED: https://github.com/calvinmetcalf/process-nextick-args/pull/9
[ ] pseudomap@1.0.2 - https://github.com/isaacs/pseudomap/pull/5
[x] regenerate@1.2.1 - master has .npmignore
[ ] resolve@1.1.7 - https://github.com/substack/node-resolve/pull/99
[ ] resumer@0.0.0 - https://github.com/substack/resumer/pull/3
[ ] rx-lite@3.1.2
[ ] rx@4.1.0
[x] slide@1.1.6 - does not contain extraneous files
[x] spdx-correct@1.0.2 - master has .npmignore
[x] spdx-exceptions@1.0.4 - no extraneous files
[x] spdx-expression-parse@1.0.2 - master has .npmignore
[ ] symbol@0.2.1 - https://github.com/seanmonstar/symbol/pull/1
[x] table@3.7.8 - master has .npmignore
[ ] text-table@0.2.0 - https://github.com/substack/text-table/pull/8
[ ] through@2.3.8 - https://github.com/dominictarr/through/pull/42
[x] time-require@0.1.2 - master has .npmignore
[ ] tmatch@2.0.1 - https://github.com/tapjs/tmatch/pull/1
[x] tunnel-agent@0.4.2 - https://github.com/request/tunnel-agent/pull/18
[x] tv4@1.2.7 - master has .npmignore
[ ] typedarray@0.0.6 - https://github.com/substack/typedarray/pull/13
[x] uid-number@0.0.6 - no extraneous files
[ ] uid2@0.0.3
[ ] util-deprecate@1.0.2
[ ] validate-npm-package-license@3.0.1 - master has .npmignore
[ ] window-size@0.1.0 - master has files entry package.json
[ ] wordwrap@0.0.3 - https://github.com/substack/node-wordwrap/pull/13
[ ] wordwrap@1.0.0
[x] wrappy@1.0.1 - https://github.com/npm/wrappy/pull/2

The list was generated with:

❯ package-config-checker | grep ✖ | sed 's/^ *✖/- [ ]/g'

It may also be worth thinking about install time as well as size. See #841 -- on my machine, AVA's install took ~50.1s. I wonder how much of that is I/O (in which case it will be fixed by addressing the size issue) vs. dependency resolution (in which case the deep nesting itself is also a cause) vs. network traffic (a bit of both, since package requests are made individually).

Another option is to pull out some of the larger deps that only cater to certain use cases into optional plugins, although I suspect the core team would prefer to avoid additional config.

The real issue here is that npm is just slow at this. npm install ava creates hundreds of http requests that just get 3XX responses. It can't even do much of it in parallel, because it needs to resolve which version of direct dependencies will be used, before it can start querying one level deep for transitive dependencies.

I don't see us dropping lots of dependencies. The only dependencies that would be easy to replace, are also ones that are not really going to cut down on download times.

Also, I really don't think comparing cleared cache download times is that valuable. I'm much more concerned with download times when we have mostly cache hits. (AVA is still pretty bad by that metric as well, but it's not as extreme).

Hello. I have been checking the list of packages, and found out that many of them already have a closed PR, specially from feross and substack authors.

I submit an updated list here with some packages edited in bold, the ones with a new PR and the ones that have an .npmignore. I also wrote the state of the related PR for each one.

The reason and previous discussions on why the PRs were not accepted can be found in feross/is-buffer#12

PACKAGE LIST

acorn-jsx@3.0.1 - master has .npmignore
amdefine@1.0.0 - master has .npmignore
argparse@0.1.16 - master has files entry in package.json
array-map@0.0.0 - substack/array-map#6 Closed
array-reduce@0.0.0 - substack/array-reduce#4 Closed
assert-plus@0.1.5 - master has .npmignore
assert-plus@0.2.0
assert-plus@1.0.0
async@0.2.10 - master has .npmignore
async@0.9.2
aws-sign2@0.5.0 - no extra files
aws-sign2@0.6.0
babel-core@6.8.0 - Also has .npmignore file
block-stream@0.0.8 - master has .npmignore
caseless@0.11.0 - request/caseless#26 Open
caseless@0.6.0
combined-stream@0.0.7 - Also has .npmignore file
combined-stream@1.0.5
commondir@1.0.1 - substack/node-commondir#6 Closed
concat-map@0.0.1 - substack/node-concat-map#5 Closed
core-js@1.2.6 - has .npmignore
core-js@2.3.0
core-util-is@1.0.2 - isaacs/core-util-is#13 Open
dashdash@1.13.0 - master has .npmignore
dashdash@1.13.1
deep-equal@0.1.2 substack/node-deep-equal#39 Closed
deep-equal@1.0.1
defined@0.0.0 - substack/defined#3 Closed
eastasianwidth@0.1.1 - komagata/eastasianwidth#3 Merged
escope@3.6.0 - master has .npmignore
esrecurse@4.1.0 - master has .npmignore
estraverse@1.9.3 - master has .npmignore
estraverse@4.1.1
estraverse@4.2.0
events-to-array@1.0.2 - isaacs/events-to-array#1 Open
extsprintf@1.0.2 - master has .npmignore
forever-agent@0.5.2 - request/forever-agent#35 Open
forever-agent@0.6.1
form-data@0.1.4 - master has .npmignore
form-data@1.0.0-rc4
formatio@1.1.1 - busterjs/formatio#8 Open
googlediff@0.1.0 - shimondoodkin/googlediff#6 Open
inflight@1.0.4 - npm/inflight#2 Closed
inherits@2.0.1 - isaacs/inherits#21 Open
is-buffer@1.1.3 - Author refuses to fix: feross/is-buffer#12 Closed
is-typedarray@1.0.0 - hughsk/is-typedarray#2 Open
isarray@0.0.1 - juliangruber/isarray#9 Open
js-yaml@3.0.1 - has files entry in package.json
json-schema@0.2.2 - kriszyp/json-schema#68 Open
jsonify@0.0.0 - substack/jsonify#6 Closed
jsonpointer@2.0.0 - janl/node-jsonpointer#26 Open
jsprim@1.2.2 - master has .npmignore
mime@1.2.11 - master has .npmignore
minimist@0.0.8 - substack/minimist#88 Closed
minimist@1.2.0
mkdirp@0.5.1 - substack/node-mkdirp#104 Closed
mute-stream@0.0.5 - npm/mute-stream#8 Open
mute-stream@0.0.6
nan@2.3.3 - master has .npmignore
node-pre-gyp@0.6.25 - master has .npmignore
npmlog@2.0.3 - npm/npmlog#32 Closed
oauth-sign@0.4.0 - request/oauth-sign#21 Merged
oauth-sign@0.8.1
optimist@0.6.1 - substack/node-optimist#142 Closed
path-is-inside@1.0.1 - has .npmignore
private@0.1.6 - benjamn/private#11 Open
process-nextick-args@1.0.6 - calvinmetcalf/process-nextick-args#9 Merged
pseudomap@1.0.2 - isaacs/pseudomap#5 Open
regenerate@1.2.1 - master has .npmignore
resolve@1.1.7 - substack/node-resolve#99 Closed
resumer@0.0.0 - substack/resumer#3 Closed
rx-lite@3.1.2 - Also has a .npmignore file
rx@4.1.0
slide@1.1.6 - does not contain extraneous files
spdx-correct@1.0.2 - master has .npmignore
spdx-exceptions@1.0.4 - no extraneous files
spdx-expression-parse@1.0.2 - master has.npmignore
symbol@0.2.1 - seanmonstar/symbol#1 Merged
table@3.7.8 - master has .npmignore
text-table@0.2.0 - substack/text-table#8 Closed
through@2.3.8 - dominictarr/through#42 Closed
time-require@0.1.2 - master has .npmignore
tmatch@2.0.1 - tapjs/tmatch#1 Open
tunnel-agent@0.4.2 - request/tunnel-agent#18 Merged
tv4@1.2.7 - master has .npmignore
typedarray@0.0.6 - substack/typedarray#13 Closed
uid-number@0.0.6 - no extraneous files
uid2@0.0.3 - coreh/uid2#17 Recently opened
util-deprecate@1.0.2 - TooTallNate/util-deprecate#6 Recently opened
validate-npm-package-license@3.0.1 - master has .npmignore
window-size@0.1.0 - master has files entry in package.json
wordwrap@0.0.3 - substack/node-wordwrap#13 Closed
wordwrap@1.0.0
wrappy@1.0.1 - npm/wrappy#2 Closed but merged as of v1.0.2

https://github.com/siddharthkp/cost-of-modules is also a nice tool:

$ cost-of-modules --no-install --less

Calculating...

┌────────────────────────────────┬──────────────┬────────┐
│ name                           │ children     │ size   │
├────────────────────────────────┼──────────────┼────────┤
│ babel-preset-stage-2           │ 57           │ 23.66M │
├────────────────────────────────┼──────────────┼────────┤
│ babel-core                     │ 35           │ 15.96M │
├────────────────────────────────┼──────────────┼────────┤
│ babel-preset-es2015-node4      │ 33           │ 11.36M │
├────────────────────────────────┼──────────────┼────────┤
│ chokidar                       │ 130          │ 5.44M  │
├────────────────────────────────┼──────────────┼────────┤
│ babel-runtime                  │ 2            │ 3.82M  │
├────────────────────────────────┼──────────────┼────────┤
│ babel-plugin-espower           │ 9            │ 2.33M  │
├────────────────────────────────┼──────────────┼────────┤
│ babel-plugin-ava-throws-helper │ 6            │ 1.18M  │
├────────────────────────────────┼──────────────┼────────┤
│ jest-snapshot                  │ 10           │ 0.87M  │
├────────────────────────────────┼──────────────┼────────┤
│ bluebird                       │ 0            │ 0.58M  │
├────────────────────────────────┼──────────────┼────────┤
│ meow                           │ 25           │ 0.30M  │
├────────────────────────────────┼──────────────┼────────┤
│ + 66 modules                   │              │        │
├────────────────────────────────┼──────────────┼────────┤
│ 76 modules                     │ 345 children │ 44.53M │
└────────────────────────────────┴──────────────┴────────┘

With npm 5.3, in a temp directory:

❯ npm init --yes
❯ npm i ava
❯ npx cost-of-modules --no-install --less
npx: installed 16 in 2.28s

Calculating...

┌───────────┬──────────────┬────────┐
│ name      │ children     │ size   │
├───────────┼──────────────┼────────┤
│ ava       │ 750          │ 46.04M │
├───────────┼──────────────┼────────┤
│ 1 modules │ 413 children │ 17.22M │
└───────────┴──────────────┴────────┘

The final size matches what Finder reports. I think that's pretty good! Plus, installation times are a lot better with npm 5.

Whilst we can always improve this, I don't think there's much use in keeping this issue open.

avajs / ava

Reduce installed package size #369

PACKAGE LIST