search

wallabyjs / public

Repository for Wallaby.js questions and issues
http://wallabyjs.com
754 stars 45 forks source link

Wallaby often hangs in VSCode #684

Closed mvestergaard closed 7 years ago

mvestergaard commented 7 years ago

This is probably a long shot, since there's no certain way to reproduce it.

I often experience that wallaby hangs (shows the progress bar [---=---]) and needs to be restarted.

It doesn't appear to happen in other editors (tried atom and webstorm).

As mentioned, there's no specific way to reproduce. It just happens after a little while when doing edits.

Here's my wallaby config file:

module.exports = function (wallaby) {
  return {
    files: [
      'src/**/*.+(ts|tsx)',
      'src/**/*.scss',
      '!src/**/*-test.+(ts|tsx)',
      'test/setup/*',
    ],

    tests: [
      'src/**/*-test.+(ts|tsx)',
    ],

    compilers: {
      '**/*.+(ts|tsx)': wallaby.compilers.typeScript()
    },

    testFramework: 'mocha',

    env: {
      type: 'node',

      params: {
        env: 'NODE_ENV=test'
      }
    },

    setup: function () {
      require('./test/setup/setup-mocha');
      require('./test/setup/setup-dom');
      require('./test/setup/setup-mobx');
    }
  }
};

setup-mocha:

'use strict';

global.chai = require('chai');
global.expect = chai.expect;
global.sinon = require('sinon');
global.__DEV__ = false;
global.BASE_URL = 'http://localhost';

chai.use(require('sinon-chai'));
chai.use(require('chai-as-promised'));

require('ignore-styles');

setup-dom:

'use strict';

const jsdom = require('jsdom');

global.document = jsdom.jsdom('<!doctype html><html><body></body></html>');
global.window = document.defaultView;
global.navigator = {
    userAgent: 'node.js'
};

if (!global.window.localStorage) {
  const store = new Map();
  global.window.localStorage = {
    getItem(key) { store.get(key); },
    setItem(key, value) { store.set(key, value); }
  };
}

propagateToGlobal(global.window);

function propagateToGlobal (window) {
  for (let key in window) {
    if (!window.hasOwnProperty(key)) continue;
    if (key in global) continue;
    if (key === 'XMLHttpRequest') continue;
    global[key] = window[key];
  }
}

global.jQuery = require('jquery');

setup-mobx:

const mobx = require('mobx');
mobx.useStrict(true);
mvestergaard commented 7 years ago

It seems like it can't figure out how to recover from errors properly or something, here i saw it happen:

screen shot 2016-07-12 at 20 09 33

You can see that there are no green indicators at the newly added test. And the progress bar is present in the status bar. It stays like this until i restart.

The errors occurred while typing.

mvestergaard commented 7 years ago

I managed to make it happen with debug enabled. Here's the log output:

[Error] RangeError: out of range index
[Error]   at RangeError (native)
[Error]   at fastCopy (/Users/mv/.vscode/extensions/WallabyJs.wallaby-vscode-1.0.19/wallaby/node_modules/ws/lib/Receiver.js:317:24)
[Error]   at Receiver.add (/Users/mv/.vscode/extensions/WallabyJs.wallaby-vscode-1.0.19/wallaby/node_modules/ws/lib/Receiver.js:78:3)
[Error]   at Socket.firstHandler (/Users/mv/.vscode/extensions/WallabyJs.wallaby-vscode-1.0.19/wallaby/node_modules/ws/lib/WebSocket.js:663:22)
[Error]   at emitOne (events.js:96:13)
[Error]   at Socket.emit (events.js:188:7)
[Error]   at readableAddChunk (_stream_readable.js:177:18)
[Error]   at Socket.Readable.push (_stream_readable.js:135:10)
[Error]   at TCP.onread (net.js:542:20)
[Error] RangeError: out of range index
[Error]   at RangeError (native)
[Error]   at fastCopy (/Users/mv/.vscode/extensions/WallabyJs.wallaby-vscode-1.0.19/wallaby/node_modules/ws/lib/Receiver.js:317:24)
[Error]   at Receiver.add (/Users/mv/.vscode/extensions/WallabyJs.wallaby-vscode-1.0.19/wallaby/node_modules/ws/lib/Receiver.js:78:3)
[Error]   at Socket.firstHandler (/Users/mv/.vscode/extensions/WallabyJs.wallaby-vscode-1.0.19/wallaby/node_modules/ws/lib/WebSocket.js:663:22)
[Error]   at emitOne (events.js:96:13)
[Error]   at Socket.emit (events.js:188:7)
[Error]   at readableAddChunk (_stream_readable.js:177:18)
[Error]   at Socket.Readable.push (_stream_readable.js:135:10)
[Error]   at TCP.onread (net.js:542:20)
[Error] RangeError: out of range index
[Error]   at RangeError (native)
[Error]   at fastCopy (/Users/mv/.vscode/extensions/WallabyJs.wallaby-vscode-1.0.19/wallaby/node_modules/ws/lib/Receiver.js:317:24)
[Error]   at Receiver.add (/Users/mv/.vscode/extensions/WallabyJs.wallaby-vscode-1.0.19/wallaby/node_modules/ws/lib/Receiver.js:78:3)
[Error]   at Socket.firstHandler (/Users/mv/.vscode/extensions/WallabyJs.wallaby-vscode-1.0.19/wallaby/node_modules/ws/lib/WebSocket.js:663:22)
[Error]   at emitOne (events.js:96:13)
[Error]   at Socket.emit (events.js:188:7)
[Error]   at readableAddChunk (_stream_readable.js:177:18)
[Error]   at Socket.Readable.push (_stream_readable.js:135:10)
[Error]   at TCP.onread (net.js:542:20)
[Error] RangeError: out of range index
[Error]   at RangeError (native)
[Error]   at fastCopy (/Users/mv/.vscode/extensions/WallabyJs.wallaby-vscode-1.0.19/wallaby/node_modules/ws/lib/Receiver.js:317:24)
[Error]   at Receiver.add (/Users/mv/.vscode/extensions/WallabyJs.wallaby-vscode-1.0.19/wallaby/node_modules/ws/lib/Receiver.js:78:3)
[Error]   at Socket.firstHandler (/Users/mv/.vscode/extensions/WallabyJs.wallaby-vscode-1.0.19/wallaby/node_modules/ws/lib/WebSocket.js:663:22)
[Error]   at emitOne (events.js:96:13)
[Error]   at Socket.emit (events.js:188:7)
[Error]   at readableAddChunk (_stream_readable.js:177:18)
[Error]   at Socket.Readable.push (_stream_readable.js:135:10)
[Error]   at TCP.onread (net.js:542:20)
[Error] Some long running code has been detected: one of your tests is taking more than 5000ms to execute.

I'm fairly sure it's not my test that is hanging since i have no async tests atm. And even so, mocha should catch that and gracefully fail.

mvestergaard commented 7 years ago

It even happens for this:

it.only('should work', function () {

});

[Error] Some long running code has been detected: one of your tests is taking more than 5000ms to execute.

May be related to the typescript compiler somehow??

ArtemGovorov commented 7 years ago

Hmm, not yet sure what's going on here. I have created this repo with the files you've listed, but can't reproduce the issue. It looks like there's something specific in your project/environment that I'm missing. Could you please clone and try if it's reproducible in your environment, and if not, please add the bits I'm missing to reproduce it (perhaps I'm missing some TS compiler settings or other packages/files that you may be using)?

There's also the possibility of the node process reuse issues caused by the used node modules caching. To exclude the possibility, could you try setting workers.recycle to false?

mvestergaard commented 7 years ago

Yea, like i said, it's difficult to reproduce. It happens intermittently. After making this issue last night, I had a fairly long period where I couldn't really make it occur, and then it started happening again.

I'll try your repo and that setting tonight, and get back to you.

I don't know if it could be related, but I've also seen a weird thing, when using nock, Inside the same test I create the nock expectation, make the http call, and validate with nock.done(). Sometimes wallaby gets into a state where these tests fail, and I have to restart to make them pass.

ArtemGovorov commented 7 years ago

It does sounds like the node process reuse issue caused by the used node modules caching. Try setting workers.recycle to false to see if it helps.

mvestergaard commented 7 years ago

Does that carry with it any particular cons? Performance overhead or something?

mvestergaard commented 7 years ago

Also I'm not sure that's the cause for the original issue, I just thought it may be related. I've seen the other thing with wallaby hanging, in code where there is no async code, or anything particular to clean up.

ArtemGovorov commented 7 years ago

With this setting wallaby starts to create node processes for each as opposed to reusing them. If the issue stops showing up, then it's definitely some project specific modules affecting it, and we may revert the setting to turn the process reuse back and try to clean just their cache like described here.

mvestergaard commented 7 years ago

I think it's fine to close this issue, if you don't find any of the errors in the logs suspicious. I'll play around with the settings you recommended and see if I can narrow down a cause.

One last unrelated question. What's going on here?

screen shot 2016-07-13 at 23 19 58

Why does the first test not display the AssertionError like the second test does?

As you can see here, it does throw a normal AssertionError

screen shot 2016-07-13 at 23 24 08
ArtemGovorov commented 7 years ago

No worries, let me know if the recycle setting helps. If it does, it may be possible to gradually clear the cache for some of your node modules and find out which one is causing the issue.

Regarding the other error, it may only happen when wallaby can't match the error stack back to the original code (to find out which line to highlight). Try printing the error stack:

try {
  ...
} catch (e) {
  console.log(e.stack);
}

and open Wallaby Tests output channel to see if there's the test file in the reported stack. If there's no, then something is corrupting the stack. Either way, it'd be awesome if you could provide the sample project I could run to see what's going on. I'll be able to find it out very quickly one I can reproduce it.

ArtemGovorov commented 7 years ago

Regarding the original issue with hanging, I have a suspicion (though the issue should be reproducible in Atom as well) about some recently introduced changes, so I rolled back wallaby.js core few versions back. Please force the core update and let me know if it works better for you.

mvestergaard commented 7 years ago

I do think I may have seen the issue in Atom at some point too, so you could be right. I just don't use Atom that often any longer.

Regarding the error reporting issue, I managed to replicate it quite easily:

This shows an error properly:

it('should show error properly', function () {
      return Promise.resolve().then(() => {
        expect(false).to.be.true;
      });
    });

This does not:

it('should show error properly', function () {
      return Promise.resolve().then(() => {
        expect({ foo: 'bar' }).to.not.deep.equal({ foo: 'bar' });
      });
    });
mvestergaard commented 7 years ago

Actually the Promise doesn't even matter. Even this doesn't show it:

it('should show error properly', function () {
  expect({ foo: 'bar' }).to.not.deep.equal({ foo: 'bar' });
});
ArtemGovorov commented 7 years ago

Hmm, I can't reproduce the issue, it displays the correct result for all 3 cases for me: screen shot 2016-07-14 at 8 23 13 pm What's your version of chai? Is it only reproducible in this project environment or if you comment out your setup function code and specify only the file with this test in your tests list the error also shows up?

ArtemGovorov commented 7 years ago

Also, is it TypeScript or it is reproducible plain JavaScript? If it is reproducible in TypeScript only, what's your TypeScript node module version and your node.js version?

mvestergaard commented 7 years ago

chai: "^3.5.0" typescript: "^2.0.0" node: 6.3.0

I'll try to see if i can replicate it in more isolation

mvestergaard commented 7 years ago

Alright i found the culprit. It's chai-as-promised somehow. This snippet should replicate it, no typescipt, otherwise same versions as above:

chai-as-promised: "^5.3.0"

const chai = require('chai');
chai.use(require('chai-as-promised'));

const expect = chai.expect;

describe('something', function () {
  it('should work', function () {
    expect({ foo: 'bar' }).to.not.deep.equal({ foo: 'bar' });
  });
});
mvestergaard commented 7 years ago 
AssertionError: expected { foo: 'bar' } to not deeply equal { foo: 'bar' }
​       at assertEql (/Users/mv/dev/js/product-designer/node_modules/chai/lib/chai/core/assertions.js:520:10)
​       at ctx.(anonymous function) (/Users/mv/dev/js/product-designer/node_modules/chai/lib/chai/utils/addMethod.js:41:25)
​       at doAsserterAsyncAndAddThen (/Users/mv/dev/js/product-designer/node_modules/chai-as-promised/lib/chai-as-promised.js:296:33)
​       at .<anonymous> (/Users/mv/dev/js/product-designer/node_modules/chai-as-promised/lib/chai-as-promised.js:255:21)
​       at ctx.(anonymous function) [as eql] (/Users/mv/dev/js/product-designer/node_modules/chai/lib/chai/utils/overwriteMethod.js:49:33)
​       at assertEqual (/Users/mv/dev/js/product-designer/node_modules/chai/lib/chai/core/assertions.js:485:19)
​       at ctx.(anonymous function) (/Users/mv/dev/js/product-designer/node_modules/chai/lib/chai/utils/addMethod.js:41:25)
​       at doAsserterAsyncAndAddThen (/Users/mv/dev/js/product-designer/node_modules/chai-as-promised/lib/chai-as-promised.js:296:33)
​       at .<anonymous> (/Users/mv/dev/js/product-designer/node_modules/chai-as-promised/lib/chai-as-promised.js:255:21)
​       at ctx.(anonymous function) [as equal] (/Users/mv/dev/js/product-designer/node_modules/chai/lib/chai/utils/overwriteMethod.js:49:33)

chai-as-promised modifies the stack even though no promises are involved. Could be the cause?

ArtemGovorov commented 7 years ago

Yep, just looking into it. The issue is that there's no user file code in the stack, so no way to understand that the error was on that specific line and highlight it. Seems like chai-as-promised issue, as wallaby doesn't modify error stacks at all.

mvestergaard commented 7 years ago

So not something you think you can fix? I doubt they will want to alter chai-as-promised to support this tool.

Either way, thanks for being so responsive Artem :)

ArtemGovorov commented 7 years ago

I'll have a look, but may not be possible to fix without changing the chai-as-promised lib. The issue does affect not just wallaby.js though, when running tests in plain mocha/karma users of the library are also not getting the error location.

ArtemGovorov commented 7 years ago

Hmm, actually disregard my previous message. The stack line is reported in plain mocha, so it must be something in wallaby that affects it, so I may actually be able to fix it.

mvestergaard commented 7 years ago

Want me to create a separate issue to track it?

ArtemGovorov commented 7 years ago

It's ok, I'll post an update here. I'm not yet sure how to fix the issue, but I found what's causing it. Wallaby.js is automatically setting chai.config.includeStack = true; and for some reason it's making the chai-as-promised to swallow stacks. For now you may just change your setup-mocha file like this:

'use strict';

global.chai = require('chai');
global.expect = chai.expect;
chai.config.includeStack = false; // <---
...

and it should work for you.

mvestergaard commented 7 years ago

Alright, thanks, will try that.

ArtemGovorov commented 7 years ago

After some playing, I was able to reproduce the original issue. The issue is with the latest node 6.3 that you may have upgraded to recently. While we are investigating it, you may install node 6.2.2, by reverting to it or possibly side by side with the latest using nvm and just specifying the node path for wallaby as described here.

ArtemGovorov commented 7 years ago

Related https://github.com/websockets/ws/issues/778

mvestergaard commented 7 years ago

Ah, yes that does look very plausible given the errors I was seeing. I'll downgrade to 6.2.2 for now. Thanks again for being so responsive. Not often you see that type of support for a commercial product :)

ArtemGovorov commented 7 years ago

Both issues, the original issue with node 6.3 support, and the missing stack/inline message issue, are fixed in core v1.0.259. Thanks for your help Mathias!