Killed during install (Ubuntu 13.10)

danielgtaylor commented 10 years ago

I'm not sure what's going on, but when trying to install on Ubuntu 13.10 I get the following:

$ npm install unicode
npm http GET https://registry.npmjs.org/unicode
npm http 304 https://registry.npmjs.org/unicode
npm http GET https://registry.npmjs.org/bufferstream
npm http 304 https://registry.npmjs.org/bufferstream
npm http GET https://registry.npmjs.org/bufferjs
npm http 304 https://registry.npmjs.org/bufferjs

> unicode@0.4.5 postinstall /home/dan/foo/node_modules/unicode
> node install.js

try to read file /usr/share/unicode/UnicodeData.txt …
parsing …
saving data as Cc.js …
saving data as Zs.js …
saving data as Po.js …
saving data as Sc.js …
saving data as Ps.js …
saving data as Pe.js …
saving data as Sm.js …
saving data as Pd.js …
saving data as Nd.js …
saving data as Lu.js …
saving data as Sk.js …
saving data as Pc.js …
saving data as Ll.js …
saving data as So.js …
saving data as Lo.js …
Killed
npm ERR! unicode@0.4.5 postinstall: `node install.js`
npm ERR! Exit status 137
npm ERR! 
npm ERR! Failed at the unicode@0.4.5 postinstall script.
npm ERR! This is most likely a problem with the unicode package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node install.js
npm ERR! You can get their info via:
npm ERR!     npm owner ls unicode
npm ERR! There is likely additional logging output above.

npm ERR! System Linux 3.8.0-19-generic
npm ERR! command "/usr/bin/node" "/usr/bin/npm" "install" "unicode"
npm ERR! cwd /home/dan/foo
npm ERR! node -v v0.10.22
npm ERR! npm -v 1.3.14
npm ERR! code ELIFECYCLE
npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     /home/dan/foo/npm-debug.log
npm ERR! not ok code 0

It looks like npm-debug.log doesn't have any more info. Any ideas? This is blocking an upgrade on a server and is quite frustrating.

dodo commented 10 years ago

hm, interesting. is this reproducible? what does ulimit -a say? can you try to install it manually with /usr/bin/time node install.js?

danielgtaylor commented 10 years ago

It is definitely reproducible on my machine, and it seems to also happen on Ubuntu 13.04. Now this is rather strange... a manual install seems to succeed after making sure dependencies are there. Even after the manual install succeeds it fails if I try it again through npm install. Maybe NPM is setting some kind of limits?

~/node-unicodetable$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 3867
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 3867
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

~/node-unicodetable$ /usr/bin/time node install.js

module.js:340
    throw err;
          ^
Error: Cannot find module 'bufferstream'
    at Function.Module._resolveFilename (module.js:338:15)
    at Function.Module._load (module.js:280:25)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at Object.<anonymous> (/home/dan/node-unicodetable/install.js:6:20)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Function.Module.runMain (module.js:497:10)
Command exited with non-zero status 8
0.07user 0.00system 0:00.08elapsed 93%CPU (0avgtext+0avgdata 10668maxresident)k
272inputs+0outputs (0major+2941minor)pagefaults 0swaps

~/node-unicodetable$ npm install
npm http GET https://registry.npmjs.org/bufferstream
npm http 304 https://registry.npmjs.org/bufferstream
npm http GET https://registry.npmjs.org/bufferjs
npm http GET https://registry.npmjs.org/buffertools
npm http 304 https://registry.npmjs.org/buffertools
npm http 304 https://registry.npmjs.org/bufferjs

> buffertools@1.1.1 install /home/dan/node-unicodetable/node_modules/bufferstream/node_modules/buffertools
> node-gyp rebuild

make: Entering directory `/home/dan/node-unicodetable/node_modules/bufferstream/node_modules/buffertools/build'
  CXX(target) Release/obj.target/buffertools/buffertools.o
  SOLINK_MODULE(target) Release/obj.target/buffertools.node
  SOLINK_MODULE(target) Release/obj.target/buffertools.node: Finished
  COPY Release/buffertools.node
make: Leaving directory `/home/dan/node-unicodetable/node_modules/bufferstream/node_modules/buffertools/build'

> unicode@0.4.5 postinstall /home/dan/node-unicodetable
> node install.js

try to read file /usr/share/unicode/UnicodeData.txt …
parsing …
saving data as Cc.js …
saving data as Zs.js …
saving data as Po.js …
saving data as Sc.js …
saving data as Ps.js …
saving data as Pe.js …
saving data as Sm.js …
saving data as Pd.js …
saving data as Nd.js …
saving data as Lu.js …
saving data as Sk.js …
saving data as Pc.js …
saving data as Ll.js …
saving data as So.js …
saving data as Lo.js …
Killed

npm ERR! unicode@0.4.5 postinstall: `node install.js`
npm ERR! Exit status 137
npm ERR! 
npm ERR! Failed at the unicode@0.4.5 postinstall script.
npm ERR! This is most likely a problem with the unicode package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node install.js
npm ERR! You can get their info via:
npm ERR!     npm owner ls unicode
npm ERR! There is likely additional logging output above.
npm ERR! System Linux 3.8.0-19-generic
npm ERR! command "/usr/bin/node" "/usr/bin/npm" "install"
npm ERR! cwd /home/dan/node-unicodetable
npm ERR! node -v v0.10.22
npm ERR! npm -v 1.3.14
npm ERR! code ELIFECYCLE
npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     /home/dan/node-unicodetable/npm-debug.log
npm ERR! not ok code 0

$ /usr/bin/time node install.js 
try to read file /usr/share/unicode/UnicodeData.txt …
parsing …
saving data as Cc.js …
saving data as Zs.js …
saving data as Po.js …
saving data as Sc.js …
saving data as Ps.js …
saving data as Pe.js …
saving data as Sm.js …
saving data as Pd.js …
saving data as Nd.js …
saving data as Lu.js …
saving data as Sk.js …
saving data as Pc.js …
saving data as Ll.js …
saving data as So.js …
saving data as Lo.js …
saving data as Pi.js …
saving data as Cf.js …
saving data as No.js …
saving data as Pf.js …
saving data as Lt.js …
saving data as Lm.js …
saving data as Mn.js …
saving data as Me.js …
saving data as Mc.js …
saving data as Nl.js …
saving data as Zl.js …
saving data as Zp.js …
saving data as Cs.js …
saving data as Co.js …
done.
5.66user 0.47system 0:06.22elapsed 98%CPU (0avgtext+0avgdata 338480maxresident)k
10672inputs+15776outputs (41major+170456minor)pagefaults 0swaps

dodo commented 10 years ago

your ulimits look fine. i cannot find Killed or exit code 137 in the nodejs code nor in the npm code. even for the node_modules grep is empty :/ I fixed a small sub-optimal peace of the code where it loads the file in js land before parsing the content. so, can you please try out 0.4.6? maybe it's really just some weird hidden ulimit which limits the memory consumption.

a different try would be rm -rf node_modules/bufferstream/node_modules/buffertools/. buffertools is a cpp addon. maybe it contains a bug or something. bufferstream will default to bufferjs, so it should still run normally.

sadly i'm unable to reproduce the error on my machine (debian sid).

dodo commented 10 years ago

another question, how much RAM has your machine?

https://ghost.org/forum/installation/1129-npm-err-weird-error-137

they had the same error because they ran out of free space.

danielgtaylor commented 10 years ago

You latest commit has fixed it for me. I think you may be right that it is a memory issue. Here's the memory on my host:

~/node-unicodetable$ free -m
             total       used       free     shared    buffers     cached
Mem:           491        216        274          0         16         55
-/+ buffers/cache:        145        345
Swap:            0          0          0

My guess is that when running npm it used just enough to hit the limit, and node install.js must have been just under the limit. Some testing of npm install shows that right now I hit around 320 megs used with the latest master, and the 0.4.5 tag gets me around 480 before it dies. Thanks for figuring this out and for pushing out a fix so quickly!

dodo commented 10 years ago

0.4.7 is optimized a little bit further.

on my machine it consumes ~70mb (~2s) now (was ~370mb (~4s) on 0.4.5 and ~225mb (~1s) on 0.4.6).

should be enough, i guess. if not feel free to reopen this bug.

thank you very much for your report.

dodo / node-unicodetable

Killed during install (Ubuntu 13.10) #8