bubble-dev / _

🍱 metarepo of many packages and various monorepos
52 stars 6 forks source link

A request for shrinking the size of @bubble-dev/start-preset #393

Closed DaleyThom closed 4 years ago

DaleyThom commented 4 years ago

Hi, @atilafassina

According to npm’s installation rule, it parses the package.json based on the libraries’s declaration order, to construct the node_modules tree. For the libraries that have not been installed in the project, node_modules tree can share them by putting them at the top level of the tree. For the libraries that have been installed, it checks whether their specified version constraints are compatible with the installed ones. If yes, npm ignores them, otherwise, npm will put such duplicated libraries in the subfolders (npm’s sandboxes).

We installed and analyzed the @bubble-dev/start-preset, and found that it is large in size. @bubble-dev/start-preset’s node_modules tree has installed 2444 libraries. 1369 out of 2444 libraries are shared at the top level of the tree, while 1075 libraries are the duplicated in the sandboxes. We further checked the duplicated libraries, and listed the them as follows (format: library name, version number, the number of installations on the node_modules tree). For example, “extend-shallow, 2.0.1, 42” denotes that library extend-shallow 2.0.1 has been installed 42 times in this project.

click for details
extend-shallow, 2.0.1, 42
kind-of, 3.2.2, 31
ms, 2.0.0, 22
semver, 5.7.1, 22
debug, 2.6.9, 22
isobject, 3.0.1, 20
source-map, 0.6.1, 17
kind-of, 6.0.3, 16
@hapi/boom, 7.4.11, 15
define-property, 0.2.5, 14
is-descriptor, 1.0.2, 13
escape-string-regexp, 1.0.5, 13
is-accessor-descriptor, 1.0.0, 13
is-data-descriptor, 1.0.0, 13
is-stream, 1.1.0, 12
define-property, 1.0.0, 12
strip-ansi, 5.2.0, 12
arr-diff, 4.0.0, 11
array-unique, 0.3.2, 11
which, 1.3.1, 11
path-exists, 4.0.0, 11
kind-of, 5.1.0, 11
is-number, 3.0.0, 11
shebang-command, 1.2.0, 10
shebang-regex, 1.0.0, 10
find-up, 4.1.0, 10
locate-path, 5.0.0, 10
ansi-styles, 4.2.1, 10
color-convert, 2.0.1, 10
expand-brackets, 2.1.4, 10
color-name, 1.1.4, 10
is-accessor-descriptor, 0.1.6, 10
has-flag, 4.0.0, 10
is-data-descriptor, 0.1.4, 10
is-descriptor, 0.1.6, 10
supports-color, 7.1.0, 10
extglob, 2.0.4, 10
fill-range, 4.0.0, 10
p-locate, 4.1.0, 10
micromatch, 3.1.10, 10
braces, 2.3.2, 10
to-regex-range, 2.1.1, 10
path-key, 2.0.1, 9
cross-spawn, 6.0.5, 8
@types/yargs, 13.0.9, 8
npm-run-path, 2.0.2, 8
@jest/types, 24.9.0, 8
ansi-regex, 2.1.1, 8
chalk, 3.0.0, 7
strip-ansi, 3.0.1, 7
pkg-dir, 4.2.0, 7
execa, 1.0.0, 7
bn.js, 4.11.9, 7
semver, 6.3.0, 7
get-stream, 4.1.0, 7
ansi-regex, 4.1.0, 7
lru-cache, 5.1.1, 6
pify, 3.0.0, 6
pify, 2.3.0, 6
@hapi/joi, 16.1.8, 6
yallist, 3.1.1, 6
is-extglob, 1.0.0, 5
supports-color, 2.0.0, 5
ansi-styles, 2.2.1, 5
chalk, 1.1.3, 5
escape-string-regexp, 2.0.0, 5
get-stream, 3.0.0, 4
is-glob, 2.0.1, 4
inherits, 2.0.3, 4
normalize-path, 3.0.0, 4
schema-utils, 2.7.0, 4
supports-color, 6.1.0, 4
anymatch, 2.0.0, 4
strip-ansi, 4.0.0, 4
readable-stream, 3.6.0, 4
emoji-regex, 7.0.3, 4
string-width, 3.1.0, 4
ssri, 7.1.0, 3
debug, 3.2.6, 3
find-cache-dir, 3.3.1, 3
ansi-regex, 5.0.0, 3
ws, 1.1.5, 3
is-wsl, 1.1.0, 3
mime, 2.4.6, 3
p-map, 3.0.0, 3
ansi-regex, 3.0.0, 3
safe-buffer, 5.2.1, 3
cliui, 5.0.0, 3
p-map, 2.1.0, 3
callsites, 3.1.0, 3
make-dir, 2.1.0, 3
wrap-ansi, 5.1.0, 3
yallist, 4.0.0, 3
ws, 6.2.1, 3
glob-parent, 3.1.0, 3
is-glob, 3.1.0, 3
ansi-styles, 3.2.1, 3
chalk, 2.4.2, 3
color-convert, 1.9.3, 3
color-name, 1.1.3, 3
has-flag, 3.0.0, 3
supports-color, 5.5.0, 3
readdirp, 3.4.0, 2
anymatch, 3.1.1, 2
source-map, 0.7.3, 2
terser-webpack-plugin, 2.3.7, 2
jest-worker, 24.9.0, 2
debug, 0.8.1, 2
json5, 1.0.1, 2
ansi-escapes, 3.2.0, 2
binary-extensions, 2.0.0, 2
cli-cursor, 2.1.0, 2
mimic-fn, 1.2.0, 2
onetime, 2.0.1, 2
restore-cursor, 2.0.0, 2
string-width, 2.1.1, 2
chokidar, 3.4.0, 2
figures, 1.7.0, 2
indent-string, 2.1.0, 2
mkdirp, 1.0.4, 2
pretty-format, 25.5.0, 2
cacache, 13.0.1, 2
glob-parent, 2.0.0, 2
cross-spawn, 5.1.0, 2
make-dir, 1.3.0, 2
yargs, 14.2.3, 2
yargs-parser, 15.0.1, 2
p-all, 2.1.0, 2
bytes, 3.1.0, 2
file-type, 5.2.0, 2
isarray, 2.0.5, 2
figures, 2.0.0, 2
fsevents, 2.1.3, 2
is-binary-path, 2.1.0, 2
slash, 3.0.0, 2
acorn, 7.3.1, 2
is-extendable, 1.0.1, 2
http-errors, 1.7.2, 2
normalize-path, 2.1.1, 2
chokidar, 2.1.8, 2
memory-fs, 0.4.1, 2

So many duplicated libraries could cause potential problems, such as: slower installs, exploding bundle sizes, and so on.

Based on npm’s installation rules, we comprehensively analyzed the compatibilities of all the transitive dependencies’ version ranges and the topological structure of your node_modules tree. We suggested @bubble-dev/start-preset add the following duplicated libraries with their compatible versions into the package.json file (as shown in the attachment), which help to refactor the project’s node_modules tree. After cleaning the local installation environment and re-installing @bubble-dev/start-preset based on the revised package.json file, we found that the node_modules tree totally installs 1998 libraries, including the 1369 ones at the top level, 629 ones in the sandboxes. As such, the revised package.json file reduces 446 duplicated libraries for @bubble-dev/start-preset. In this manner, reducing 2634 files (from 43919 to 41285) and 644 folders (from 6618 to 5974) in total. Note that, all the added directed dependencies in package.json file are deduced based on the original node_modules tree, which will not miss any required dependencies.

click for details
"semver": "5.6.0",
"source-map": "0.6.1",
"extend-shallow": "2.0.1",
"debug": "2.6.9",
"isobject": "3.0.1",
"is-stream": "1.1.0",
"ansi-styles": "4.1.0",
"path-exists": "4.0.0",
"supports-color": "7.1.0",
"find-up": "4.1.0",
"which": "1.2.9",
"ansi-regex": "2.0.0",
"array-unique": "0.3.2",
"shebang-command": "1.2.0",
"is-number": "3.0.0",
"strip-ansi": "3.0.1",
"kind-of": "6.0.2",
"is-extendable": "1.0.1",
"define-property": "1.0.0",
"path-key": "2.0.1",
"braces": "2.3.2",
"pkg-dir": "4.2.0",
"arr-diff": "4.0.0",
"is-descriptor": "0.1.0",
"@hapi/joi": "16.0.0",
"cross-spawn": "6.0.5",
"micromatch": "3.1.10",
"@hapi/boom": "7.4.11"

Would you like to adopt this suggestion to shrink the project’s size?

Thank you for your attention. Best regards. package.zip package-lock.zip