slimerjs hangs and refuses to exits

[lbrewington]

I have a project where I am using slimerjs because of a requirement to utilize flash to work with a website that requires interaction with an adobe flex application.

I am having trouble only with this url, as the request processes but slimmerjs refuses to release the process. I have stepped down through various setups and versions as briefly outlined below.

• base phantomjs
• phantomjs with casperjs
• slimerjs with casperjs via xfvb
• slimerjs with casperjs via ubuntu-desktop (lightdm / gnome-shell)
• slimerjs vai ubuntu-desktop

I think this is an important piece of information because I can physically see slimerjs running.

I have used a number of versions and execution methods, but I have tried to narrow this problem down. In this regard the version I am using right now is :

  Innophi SlimerJS 0.10.0pre, Copyright 2012-2014 Laurent Jouanneau & Innophi

So here is the code I am using. I pulled this from the getting started slimmer docs

var page = require("webpage").create();
page.open("http://myapps.paychex.com")
//page.open("http://github.com")
    .then(function(status){
        if (status == "success") {
            console.log("The title of the page is: "+ page.title);
        }
        else {
            console.log("Sorry, the page is not loaded");
        }
        page.close();
        phantom.exit();
        slimerjs.exit();
        slimer.exit();
    })

In this code, the console.log of the page works, but no execution takes place after that point. Can you give me any pointers on what I might be doing wrong, or why slimerjs refuses to release the process.

Thanks in advance,

and here is the output (debug) mode that it being output.

http://pastebin.com/VgNC4DqS

[lbrewington]

Update : If I remove page.close(); code block and just utilize slimer.exit(), it seems to release the process every time. Is page.close() necessary, as this appears to be where the execution is hanging.

I eventually would like to link this up with casperjs, and I think casperjs is automating page.close() into the workflow.

Update 2 : I take back what I said about page.close(). I think the exit() just executes prior to the whatever is causing the bug. IE, it exits before doing all the redirects and processing all the pages.

[laurentj]

slimerjs.exit(); does not exist. only slimer.exit();

I tried your script, it works well with me. No hang.

[lbrewington]

Thanks for looking at this, but it is still causing me problem. Could you please look at this second time and follow my comments below. Very much appreciated.

The script works for me too, at first. Then it stop responding after that, which is odd to say the least.

Just this morning I ran the slimerjs script from the command line. As expected, the windows appear in the UI and then the command prompt outputs the title of the page with no hang. I ran it again, and received expected behavior with no hang.

After the second time, it starts hanging. The weird thing is that the windows open and the command prompt outputs the title of the page. And then it just hangs. The mini firefox browser closes but the slimerjs window stays open. Sometimes the script completes, sometimes it hangs.

Any ideas and what might be causing the hang, or strategies for tracking down the problem?

[DarrenCook]

Re: strategies to narrow it down: you could try different versions of Firefox. Or start up a few different distros on EC2 (or wherever) to see if that narrows it down. Also try from different regions, as perhaps this site returns different HTML based on IP sniffing.

Your script works fine for me too, on my notebook (Mint 17), from U.K., using SlimerJS 0.9.1.

(The only change I made to your script was to delete phantom.exit(); and slimerjs.exit(); and just leave slimer.exit())

[lbrewington]

Okay, I think increasing the memory to use 4GB and 4 Processor is one solution that prevents the hang. Of course, my fear is that hangs will happen undetected, which could be a disaster since I could only assume complete failure. Also, this is a very simple script and the end script would contain many many more instructions.

Still don't understand why the program would just hang if it was low memory situation.

At any rate, I'm going to continue to throw hardware at the problem and see if it can become consistent. If that works I should be good to go. I also need to convert this into a working casperjs setup and test with that and see if I can achieve the same results.

Thanks for you help and taking the time to look into this.

[lbrewington]

Spoke too soon, intermittently hanging still after a number of runs.

[lbrewington]

Could this be insight into the problem: (taken from --jsconsole)

Timestamp: 10/13/2014 02:02:35 PM Error: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIObserverService.removeObserver] Source File: resource://slimerjs/addon-sdk/toolkit/loader.js -> resource://slimerjs/slimer-sdk/webpage.js Line: 715

[g0morra]

Are you still testing with slimerjs.exit() in there? In my experience Slimer will hang if you have an issue with unitialized variables being called to do something, trying to modify properties of objects that haven't been instantiated etc. So if the script is only intermittently failing, my guess would be that sometimes the phantom.exit() is exiting cleanly, but other times the JS interpreter is calling slimerjs.exit() before slimer has exited cleanly, causing a hang because slimerjs has not been declared.

slimer.exit() is inprecise in that the script can continue running for a couple of seconds while xulrunner is shutting down. It's just the async nature of javascript. If you want to do a "hard" stop and prevent all further activity after you call slimer.exit(), then make your down exit function, wrapping slimer.exit() and setting a global variable to indicate that all activity should stop. Then wrap all of the action in promises, and inside each promise check for your global die variable: for example in each promise, if(dieNow === true) return false, otherwise perform the action you want to in that promise. Inside my own die function, I do other cleanup tasks like making a final screenshot of the desktop with Imagemagick etc.

You can try using "use strict" which will throw an error on uninitialized variables, rather than difficult to debug silent errors.

You only need slimer.exit(), the other stuff is unnecessary. If you take a look at the source, phantom.exit() does exactly the same thing as slimer.exit(). You don't need to close pages before you exit slimer.

[g0morra]

Thanks Laurent, very cool.

[lbrewington]

Just so you know, I am not using the invalid syntax. I removed the erroneous code used as soon as I realized it was invalid syntax (was never really using it anyway), and honestly I don't think it is affecting the situation at all. If I switch the URL to load to another website and run any script I never have a problem. The problem is only ever introduced if I use the URL myapps.paychex.com.

Another interesting observation is that if I supply the URL and immediately exit, there is no problem. This is because the page is using a number of technologies to drive a single site login (flash, angular, template loading, etc). However, if I introduce a wait period using this code

    this.wait(5000);

slimerjs once again hangs and refuses to release the process. If you look at that url, you can see what it is necessary to wait, there are quite a few redirection and iframe loads going on. Something in that loading is causing a problem I believe.

Also, the script sometimes works and sometimes doesn't. When I say doesn't work, I mean the process continues to run and the slimerjs window never closes until I ctrl + c.

I can only guess to the problem. There is a redirect that is somehow failing. There is a redirect response code not playing well. There is a slimmer js observer that is waiting on an event that will never take place. I don't know, and I don't know how to find out what that process is doing behind the scenes.

I'm at wits end on this one. I've installed a fresh Ubuntu iso to a virtual and I have downloaded the nightly build of slimerjs (the 0.9.4pre ...and I've also been working with the 0.10.0pre branch as well). The syntax of my code is solid (and very very simple to boot) and if I attempt to parse another website I never have a problem. The real head scratcher is that sometimes the code returns as expected, but other times just sits in limbo, which indicates to me there is some sort of race condition.

I appreciate the help and I'm continuing to get to the bottom of the problem, so any insight you could give as to why this particular URL does not want o play well would be very appreciated.

[bernardeli]

I've got trapped into a similar issue.

After debugging, I had realised that it was due to permission issues whereby firefox profiles couldn't get accessed. It seems to rely on ~/.innophi, ~/.mozilla & ~/.cache. Make sure you get the right permissions for them. I ended up removing them so that slimerjs + firefox would recreate automatically with the right permissions set.

The way I found was by forcing the error at https://github.com/laurentj/slimerjs/blob/master/src/slimerjs#L174-L178 to always show (the else condition).

Hope that helps. Cheers

[laurentj]

Similar issues: karma-runner/karma-slimerjs-launcher#1 karma-runner/karma-slimerjs-launcher#3

[laurentj]

It seems it is a blocker for Mozilla https://bugzilla.mozilla.org/show_bug.cgi?id=1125093