gotwarlost / istanbul

Yet another JS code coverage tool that computes statement, line, function and branch coverage with module loader hooks to transparently add coverage when running tests. Supports all JS coverage use cases including unit tests, server side functional tests and browser tests. Built for scale.
Other
8.7k stars 787 forks source link

Crashes randomly when running with Mocha #765

Closed serge1peshcoff closed 7 years ago

serge1peshcoff commented 7 years ago

I am using Mocha to write tests and Istanbul to count code coverage. And I am using this script to run Mocha with Istanbul to count code coverage and to post it to Github:

istanbul cover ./node_modules/mocha/bin/_mocha --report lcov -- -R spec --timeout 10000 && cat ./coverage/lcov.info | ./node_modules/coveralls/bin/coveralls.js

The thing is, if I run my tests without Istanbul, with mocha --timeout 10000, everything works fine, but if I run this command above, it crashes sometimes. I tried running this command 5 times and it crashed in 3 cases (when I was running mocha, everything worked every time I ran it).

Here is my npm-debug.log which was created after the crash:

1 verbose cli [ '/home/serge1peshcoff/.nvm/versions/node/v7.0.0/bin/node',
1 verbose cli   '/home/serge1peshcoff/.nvm/versions/node/v7.0.0/bin/npm',
1 verbose cli   'run',
1 verbose cli   'test' ]
2 info using npm@3.10.8
3 info using node@v7.0.0
4 verbose run-script [ 'pretest', 'test', 'posttest' ]
5 info lifecycle oms-events@0.1.0~pretest: oms-events@0.1.0
6 silly lifecycle oms-events@0.1.0~pretest: no script for pretest, continuing
7 info lifecycle oms-events@0.1.0~test: oms-events@0.1.0
8 verbose lifecycle oms-events@0.1.0~test: unsafe-perm in lifecycle true
9 verbose lifecycle oms-events@0.1.0~test: PATH: /home/serge1peshcoff/.nvm/versions/node/v7.0.0/lib/node_modules/npm/bin/node-gyp-bin:/home/serge1peshcoff/aegee-docker/oms-docker/oms-events/node_modules/.bin:/home/serge1peshcoff/.nvm/versions/node/v7.0.0/bin:/home/serge1peshcoff/.rbenv/shims:./node_modules/.bin:/home/serge1peshcoff/.rbenv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/lib/jvm/java-8-oracle/bin:/usr/lib/jvm/java-8-oracle/db/bin:/usr/lib/jvm/java-8-oracle/jre/bin:/home/serge1peshcoff/.rvm/bin:/home/serge1peshcoff/Android/Sdk/tool
10 verbose lifecycle oms-events@0.1.0~test: CWD: /home/serge1peshcoff/aegee-docker/oms-docker/oms-events
11 silly lifecycle oms-events@0.1.0~test: Args: [ '-c',
11 silly lifecycle   'istanbul cover ./node_modules/mocha/bin/_mocha --report lcov -- -R spec --timeout 10000 && cat ./coverage/lcov.info | ./node_modules/coveralls/bin/coveralls.js' ]
12 silly lifecycle oms-events@0.1.0~test: Returned: code: 1  signal: null
13 info lifecycle oms-events@0.1.0~test: Failed to exec test script
14 verbose stack Error: oms-events@0.1.0 test: `istanbul cover ./node_modules/mocha/bin/_mocha --report lcov -- -R spec --timeout 10000 && cat ./coverage/lcov.info | ./node_modules/coveralls/bin/coveralls.js`
14 verbose stack Exit status 1
14 verbose stack     at EventEmitter.<anonymous> (/home/serge1peshcoff/.nvm/versions/node/v7.0.0/lib/node_modules/npm/lib/utils/lifecycle.js:255:16)
14 verbose stack     at emitTwo (events.js:106:13)
14 verbose stack     at EventEmitter.emit (events.js:191:7)
14 verbose stack     at ChildProcess.<anonymous> (/home/serge1peshcoff/.nvm/versions/node/v7.0.0/lib/node_modules/npm/lib/utils/spawn.js:40:14)
14 verbose stack     at emitTwo (events.js:106:13)
14 verbose stack     at ChildProcess.emit (events.js:191:7)
14 verbose stack     at maybeClose (internal/child_process.js:877:16)
14 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:226:5)
15 verbose pkgid oms-events@0.1.0
16 verbose cwd /home/serge1peshcoff/aegee-docker/oms-docker/oms-events
17 error Linux 4.8.0-37-lowlatency
18 error argv "/home/serge1peshcoff/.nvm/versions/node/v7.0.0/bin/node" "/home/serge1peshcoff/.nvm/versions/node/v7.0.0/bin/npm" "run" "test"
19 error node v7.0.0
20 error npm  v3.10.8
21 error code ELIFECYCLE
22 error oms-events@0.1.0 test: `istanbul cover ./node_modules/mocha/bin/_mocha --report lcov -- -R spec --timeout 10000 && cat ./coverage/lcov.info | ./node_modules/coveralls/bin/coveralls.js`
22 error Exit status 1
23 error Failed at the oms-events@0.1.0 test script 'istanbul cover ./node_modules/mocha/bin/_mocha --report lcov -- -R spec --timeout 10000 && cat ./coverage/lcov.info | ./node_modules/coveralls/bin/coveralls.js'.
23 error Make sure you have the latest version of node.js and npm installed.
23 error If you do, this is most likely a problem with the oms-events package,
23 error not with npm itself.
23 error Tell the author that this fails on your system:
23 error     istanbul cover ./node_modules/mocha/bin/_mocha --report lcov -- -R spec --timeout 10000 && cat ./coverage/lcov.info | ./node_modules/coveralls/bin/coveralls.js
23 error You can get information on how to open an issue for this project with:
23 error     npm bugs oms-events
23 error Or if that isn't available, you can get their info via:
23 error     npm owner ls oms-events
23 error There is likely additional logging output above.
24 verbose exit [ 1, true ]

How can I fix this?

serge1peshcoff commented 7 years ago

Crap, that was totally my fault.

In case anyone stumbles upon the same error, here is the code that caused the crashing (to be more precise, exiting) and how to fix it:

process.on('uncaughtException', (err) => {
  log.error(err);
  // If something goes wrong, the process will exit.
  // In testing environment, on some exception Mocha
  // would just exit without printing anything. So I added
  // this checking to fix it.
  if (process.env.NODE_ENV !== 'test') {
    process.exit(1);
  }
});

That line of code made Mocha just stop the process, not finishing the test and the report.

ORESoftware commented 7 years ago

yeah Mocha is not great software. I believe it uses a global handler like this and assigns errors to test cases in the global scope. I recommend moving to AVA or something else.

WoodySlum commented 7 years ago

Hey ! I have exactly similar issue, but I didn't understand where should I put the piece of code. Could you help me ?

serge1peshcoff commented 7 years ago

@WoodySlum my issue was that I was using custom process.on('uncaughtException') and process.exit() was run inside it, so the Mocha would exit silently instead of writing a report. So I've wrapped the exit call inside a if statement that checks if the process.env is not test