search

meteor / todos

The example app "Todos", written following the Meteor Guide
Other
535 stars 366 forks source link

[1.4.2.1-rc.0] meteor todos client tests fail with select: Invalid argument #198

Open hwillson opened 7 years ago

hwillson commented 7 years ago

Migrating this over from meteor/meteor#8016.

(originally reported by @hexsprite)

Noticed this in my project and was able to reproduce using the meteor-todos project.

Just upgrade to rc.0 and you'll see that when running npm test the client tests fail. In fact, it seems the process exits unceremoniously somehow since the output is cut off.

I20161104-09:06:23.738(-7)? --------------------------------
I20161104-09:06:23.738(-7)? ----- RUNNING CLIENT TESTS -----
I20161104-09:06:23.738(-7)? --------------------------------
I20161104-09:06:23.738(-7)? 
I20161104-09:06:23.738(-7)? select: Invalid argument
....  80 repeated snipped ....
I20161104-09:06:23.738(-7)? select: Invalid argu
➜  todos git:(master) ✗ echo $?
0
hwillson commented 7 years ago

I can't seem to re-create this issue with Meteor 1.4.2.3 (OSX 10.11.5). Issue meteor/meteor#8016 had a fair number of up votes though, so if anyone is able to reproduce this issue with the current version of todos please let me know. For now I'll consider this issue closed, but let me know if it should be re-opened. Thanks!

dburles commented 7 years ago

Strangely, replacing the factory package caused this exact issue! 😕

hwillson commented 7 years ago

Interesting - I'll take a look - thanks!

weilitao commented 7 years ago

The bug is still exists. Todo app and my app all have the same issue when running npm test, npm test-watch is ok.

OSX 10.12.1 Meteor 1.4.2.3 phantomjs 2.1.1

hwillson commented 7 years ago

I've looked into this a bit further. This doesn't appear to be a dispatch:mocha-phantomjs issue, like DispatchMe/meteor-mocha-phantomjs#33 suggests. I'd like to say it's a phantomjs issue (something like ariya/phantomjs#13231), but I'm not 100% convinced of that yet either. I've been debugging the dispatch:phantomjs-tests package to get access to the original phantomjs process while running tests, and haven't quite found the real issue yet. A few troubleshooting notes:

I'll look into this more shortly.

robsecord commented 7 years ago

+1 Experiencing the same issue when upgrading to Meteor 1.4.2.3 from 1.4.2 (OSX 10.12.1)

awatson1978 commented 7 years ago

I've been debugging this all day, and am of the opinion that the number of packages in the .meteor/packages directory can semi-reliably reproduce the error.

We have a test harness for running QA scripts across the Clinical Meteor distro, and the way the harness and packages are configured, we have about 40 small packages, each with a different SimpleSchema in it. Anyhow, I can reliably reproduce the error with different combinations of packages, which are all effectively exactly the same. It happens consistently when I add the 47th package.

I was thinking "that's totally random, and couldn't be related to the Todos example'. But then I looked, and Todos actually has 49 packages. Probably one or two of them somehow aren't getting loaded during the meteor test command.

Anyhow, some sort of buffer overflow. Could be amount of memory the packages are using; number of files on disk; time it takes to load them, etc. But PhantomJS doesn't like it when there are too many packages, and burps this select: Invalid argument error.

gsabran commented 7 years ago

For me replacing dispatch:mocha-phantomjs by dispatch:mocha fixed the issue

awatson1978 commented 7 years ago

Unfortunately, it dispatch:mocha doesn't run the tests on client. :/

aldeed commented 7 years ago

Try with dispatch:mocha-phantomjs@0.1.9, which uses latest phantomjs. Seems to have fixed this for me in another app, but it's possible that something else I did fixed it.

hwillson commented 7 years ago

Thanks @aldeed - updating to dispatch:mocha-phantomjs@0.1.9 didn't help unfortunately.

@awatson1978 - I definitely think you're on to something! I looked into why meteor npm test always works after running a meteor reset. Instead of running a full reset, if you just delete your apps .meteor/local/plugin-cache it will work also. I tested just removing some of the packages in the plugin-cache directory, and testing failed; I then removed most of them (but left about 40) and it worked. Very interesting!

For those who want a really quick workaround (hack), run your tests as (from your app root):

rm -Rf .meteor/local/plugin-cache; meteor npm test

Needless to say this is quite slow (since we're killing the plugin cache), but it works until we can get a better fix in place.

iMagdy commented 7 years ago

Clearing cache didn't work for me unfortunately. dispatch:mocha-phantomjs@0.1.9

timbrandin commented 7 years ago

I have the same issue, also the cache is always cleared when we run it fresh on a test docker.

hwillson commented 7 years ago

@iMagdy @timbrandin Sorry to hear that! We definitely still need to find the source of the memory leak. I'll try to look into this more today. Thanks!

timbrandin commented 7 years ago

For me it's not a huge ordeal, we just started adding tests and we're not depending on them yet. But thx for looking into it so soon.

timbrandin commented 7 years ago

Shouldn't this be solved in: https://github.com/DispatchMe/meteor-mocha-phantomjs/issues/33 ?

hwillson commented 7 years ago

Quite possibly, but I haven't pinpointed the source of the problem yet. I know phantomjs is running out of memory, but I haven't been able to track down what's causing that to happen. It's quite possible some combination of things that the todos app exposes is causing that leak, so it's difficult to move this issue over to the https://github.com/DispatchMe/meteor-mocha-phantomjs repo until we know more.

hwillson commented 7 years ago

Hi all - I recently discussed this issue with @abernix and be brought up some good points about the stability of phantomjs v2.x, and mentioned trying the last 1.9.x version. I just tried replacing the 'phantomjs-prebuilt': '2.1.13' dependency in meteor-phantomjs-tests with 'phantomjs': '1.9.19', and everything works properly again. I'll try opening a new issue about this in the phantomjs repo, but with 1666 open issues and an approximate issue resolution time of 56 days, I'm really not optimistic we'll get a timely response.

In the meantime, I'll package up a forked version of dispatch:mocha-phantomjs that people can drop in to verify that this does indeed fix the issue. Once verified, we can put some more through into how to address this moving forward. Thanks!

abernix commented 7 years ago

I don't recall if the issue I was running into was the same or not, but I spent a lot of time debugging it. My comment on the (private) commit was:

Phantom 2 just seems to evalulate the page before it's ready.  It might be a problem with Qt, but I don't have time to check.

As you can see by my misspelling of "evaluate", I didn't have much time but had got so far as to thinking it was a problem with Qt, a PhantomJS dependency. Ultimately, I moved to 1.9.x as well and have been there ever since.

JoshuaColeman commented 7 years ago

I tested downgrading meteor-phantomjs-tests to 'phantomjs': '1.9.19' here: https://github.com/DispatchMe/meteor-phantomjs-tests/compare/master...JoshuaColeman:downgrade-phantomjs

The result (on OSX) was that client tests reported running without any output, and an 'undefined' number of errors. I added some extra logging around the call to phantomjs and found that the 1.9.19 phantomjs crashes with a SIGSEV fault. I tested directly calling phantomjs and sure enough: Segmentation fault: 11

This is a known issue documented here: https://github.com/ariya/phantomjs/issues/14558

Any other ideas on how to resolve this?

hwillson commented 7 years ago

Ouch - I just updated to macOS Sierra and now it's failing for me as well. Using phantomjs 1.9.19 worked for me on El Capitan, but it now looks like this is another dead end. Back to the drawing board ...

NickBusey commented 7 years ago

meteor reset seems to be quite a reliable fix for me. To the point that editing the test line in package.json to "test": "meteor reset; meteor test --once --driver-package dispatch:mocha-phantomjs",, while far from ideal, does work repeatably.

0o-de-lally commented 7 years ago

See fix: https://github.com/meteor/meteor/issues/8225. Updating to meteor update --release 1.4.3-rc.3 seems to fix this. Also in https://github.com/DispatchMe/meteor-mocha-phantomjs/issues/33

hwillson commented 7 years ago

Woohoo - awesome news @keyscores! Thanks for testing it out and posting back here!

0o-de-lally commented 7 years ago

Sadly updating to dispatch:mocha-phantomjs@=0.1.9 causes the error to return on my machine. Though 0.1.7 works on my local. @hwillson can you confirm?

benjamn commented 7 years ago

@keyscores Since meteor update --release 1.4.3-rc.xxx doesn't permanently change the default version of Meteor, it's easy to end up using the latest official version (1.4.2.6) by accident. Are you sure you're running with meteor --release 1.4.3-rc.4 ... (or in an app that has METEOR@1.4.3-rc.4 in its .meteor/release file)?

0o-de-lally commented 7 years ago

@benjamn Yep I'm pretty sure I'm at the right version. Here are my steps:

// ## passing ##
> unzip todos-master.zip 
> cd todos-master
> meteor npm install
> meteor update --release 1.4.3-rc.4
> cat .meteor/release 
// METEOR@1.4.3-rc.4
> meteor list 
// dispatch:mocha-phantomjs          0.1.7
> meteor test --full-app --once --driver-package dispatch:mocha-phantomjs --release 1.4.3-rc.4
// OK tests passing smoothly

// ## failing ##
> meteor update dispatch:mocha-phantomjs 
// dispatch:mocha-phantomjs  upgraded from 0.1.7 to 0.1.9
// dispatch:phantomjs-tests  upgraded from 0.0.5 to 0.0.7
> meteor test --full-app --once --driver-package dispatch:mocha-phantomjs --release 1.4.3-rc.4
// select: Invalid argument 
// select: Invalid argument 
// continues 200x
aldeed commented 7 years ago

The only real difference is the bump from using phantomjs-prebuilt 2.1.7 to 2.1.13, but I don't actually think that phantomjs has fixed select: Invalid argument. I think it appears randomly sometimes and other times not, possibly related to memory or number of packages as others have suggested. If you rerun with both versions multiple times, I'm guessing you'll see it once in a while regardless of version.

0o-de-lally commented 7 years ago

@aldeed I've run through the steps above again. It's very consistent. I'm not exactly sure what is being lost from upgrading from dispatch:mocha-phantomjs 0.1.7 to 0.1.9 . Have you been able to reproduce?

0o-de-lally commented 7 years ago

@benjamn have you been able to reproduce my steps above?

derouck commented 7 years ago

I bumped into this "select: Invalid argument" error as well for a different usecase (PDF rendering). But the solution I found might be interesting. The problem occurred for me when there were images loaded from the same server (src="Meteor.absoluteUrl()/....). Earlier I didn't have problems with that case, but with latest Meteor update this broke the PDF rendering.

I could solve it by making the path to the image relative and by setting up a base path for these files in the phantom.js settings. I have no idea if this finding is relevant for the tests issue, maybe it helps to point you in the right direction?

lorensr commented 7 years ago

Issue consistently persists for me in:

macOS@10.12.3
meteor@1.4.3.2
dispatch:mocha-phantomjs@0.1.7 – 0.1.9
> git clone git@github.com:meteor/todos.git
> meteor update --release 1.4.3.2
> meteor npm install
> meteor list | grep phantom
dispatch:mocha-phantomjs          0.1.7

> meteor test --full-app --once --driver-package dispatch:mocha-phantomjs
PASS

> meteor test --full-app --once --driver-package dispatch:mocha-phantomjs
FAIL (and fails subsequent times)

> rm -Rf .meteor/local/plugin-cache
> meteor test --full-app --once --driver-package dispatch:mocha-phantomjs
PASS (and fails subsequent times)

Same behavior after meteor update dispatch:mocha-phantomjs to 0.1.9.

aldeed commented 7 years ago

We need to update todos app to use the latest dispatch:mocha instead of dispatch:mocha-phantomjs, and use one of the other supported browsers instead of phantomjs. dispatch:mocha-phantomjs is deprecated. Phantom is just too broken. ~Any volunteers?~

Edit: Scratch that, just noticed the commit from @lorensr. Cool!

NickBusey commented 7 years ago

I made similar changes to my app, and testing seems to work fine now. I'd say this is closed.

robsecord commented 7 years ago

For our test environment, Nightmare works well for the developers who are running MacOS. However, we run into problems when using Nightmare with our Linux-based Dev and CI servers, in which case we need to run Phantom in order to have our tests pass. This creates an inconsistent testing environment between Dev and CI. For now, we are using such a workaround, but I would not consider this issue closed.

hwillson commented 7 years ago

Merging #222 auto-closed this issue, but https://github.com/meteor/todos/issues/198#issuecomment-292586439 still needs to be investigated.