NPM package is bundled with useless files and is too big (3 Mo that can be reduced to 620 Ko)

[yvele]

First of all: NeDB is awesome!

Big NPM package with unnecessary files

The NPM package is quite big and includes many unnecessary files (and dependencies) for a Node.js usage.

Uncompressed NPM package size is about 3 Mo and can be reduced to 620 Ko !

Here is the total size with dependencies:

 88K    async
132K    binary-search-tree
1.1M    localforage         (UNNECESSARY)
 92K    mkdirp
1.4M    nedb
184K    underscore

And the detailed size of naked NeDB package

4.0K    LICENSE
 44K    README.md
 44K    benchmarks          (USELESS)
4.0K    bower.json          (USELESS)
916K    browser-version     (USELESS)
4.0K    index.js
 96K    lib
4.0K    package.json
292K    test                (USELESS)
 20K    test_lac            (USELESS)

Potential Solutions

I'm focused on the NPM deployment so I may miss some browser deployment considerations.. Let me know if it make sense..

.npmignore ?

I think NeDB can easily save 1.30 Mo by adding the following .npmignore file to the repo:

benchmarks
bower.json
browser-version
test
test_lac

PS: This has already been done in the bower.json file.

Move localforage to devDependencies ?

Now about the localforage dependency.. it looks like it is already bundled into browser-version/out.. Can we simply move localforage dependencies to devDependencies in package.json?

[kaizhu256]

there's a single-script version of nedb @ https://www.npmjs.com/package/nedb-lite that has zero npm-dependencies.

it tracks the latest nedb version (v1.8.0) and is essentially one file (the standalone browser script with the storage component slightly modified so it can run in nodejs as well).

i use i for my project https://www.npmjs.com/package/swagger-lite, and it works fine in both browser and nodejs.

[yvele]

@kaizhu256 That's nice. But I still think NeDB could be improved it self to be lightweight with minimum efforts. What do you think can be done to NeDB to improve it's size, based on your experience with nebd-lite?

[louischatriot]

Thanks for the kind words :)

So there are also standalone versions of NeDB for the browser which clock in at 292KB and 98 KB when minified: https://github.com/louischatriot/nedb/blob/master/browser-version/out/nedb.js and https://github.com/louischatriot/nedb/blob/master/browser-version/out/nedb.min.js

That said you are right about the server version, the npm package itself could be cleaned to only include the necessary files. Will try to do it when I have the time!

[yvele]

@louischatriot Yes I'm only talking about the NPM server version in this issue ;) The server package size is less critical than the browser size, but it's still a problem.

[kaizhu256]

@yvele this is the gist of nedb-lite

--- /Users/kaizhu/tmp/nedb.js   2016-05-04 16:42:04.000000000 +0300
+++ nedb-lite.js    2016-05-04 16:40:44.000000000 +0300
@@ -2990,7 +3251,7 @@
-var storage = require('./storage')
+var storage = local.storage = local.storage || require('./storage')

the above one-line change allows nedb's browser script to use nodejs's filesytem if its available (plus an extra 300 sloc to implement local.storage for nodejs).

i've attached the full diff between nedb-lite.js and the browser nedb.js for the gory details (diff.txt)

diff.txt

[bahmutov]

You can whitelist the files quickly in package.json and check the published size, see https://glebbahmutov.com/blog/smaller-published-NPM-modules/ for example to measure NPM published size you can add script command

"scripts": {
  "size": "t=\"$(npm pack .)\"; wc -c \"${t}\"; tar tvf \"${t}\"; rm \"${t}\";"
}

This will print the list of files included in published tar as well as total size