Error 403: Directory Listing Denied kills PhantomJS instance

[MaestroJurko]

Getting this issue randomly. Does anyone know what could be the reason for this error?

Unhandled rejection SyntaxError: Unexpected token E at Object.parse (native) at IncomingMessage. (/vagrant/node_modules/node-phantom-simple/node-phantom-simple.js:459:28) at emitNone (events.js:72:20) at IncomingMessage.emit (events.js:166:7) at endReadableNT (_stream_readable.js:905:12) at nextTickCallbackWith2Args (node.js:478:9) at process._tickCallback (node.js:392:17) Unhandled rejection SyntaxError: Unexpected token E at Object.parse (native) at IncomingMessage. (/vagrant/node_modules/node-phantom-simple/node-phantom-simple.js:459:28) at emitNone (events.js:72:20) at IncomingMessage.emit (events.js:166:7) at endReadableNT (_stream_readable.js:905:12) at nextTickCallbackWith2Args (node.js:478:9) at process._tickCallback (node.js:392:17)

[MaestroJurko]

Ok, found the problem. Issue is in parsing data, because data is this:

{ [HeadlessError: Error parsing JSON from phantom: SyntaxError: Unexpected token E
Data from phantom was: Error 403: Directory Listing Denied
Directory listing denied]
  name: 'HeadlessError',
  message: 'Error parsing JSON from phantom: SyntaxError: Unexpected token E\nData from phantom was: Error 403: Directory Listing Denied\nDirectory listing denied' }

[MaestroJurko]

This error also kills PhantomJS instance, which is not ok.

[MaestroJurko]

In bridge.js, watchdog_clear should not kill phantomjs instance, but close the page and if there is no page opened, then close the instance.

[ohenepee]

Please what is causing the 403 Directory Listing Denied and can it be solved?

As to what causes it is unknown to me, but I am aware it comes from the Mongoose server... is the server buggy? Or is there some timing issues that need to be addressed (as in rapid repetitive / intermittent launching of phantomjs executable)

[ohenepee]

I found a way to reproduce the bug... please run the code for 20 mins and you'll see the bug pop up. I ran mine on a VPS with Ubuntu Server 14.04 LTS 64-bit

const Horseman = require("node-horseman");

var browserIsReady = true;

function reachPage() {

  var horseman = new Horseman();
  console.log("Starting...");

  horseman
  .viewport(800, 600)
  .open("https://www.facebook.com/bbcnews")
  .status()
  .then(function (status) {
    if (Number(status) != 200) {
      console.log("Couldn't load page, trying again...");
      return horseman.close();
    }
  })
  .wait(1e4)
  .log("Stepping out...") // prints out the durl
  .wait(3e3)
  .finally(function() {
    horseman.close();
    setTimeout(reachPage, 3e3);
    return
  });

}

reachPage();

[sbryfcz]

Any status on this bug? Hitting the same issue.

[baudehlo]

This doesn't seem like a bug - more that the provider is throttling you, no?

On Wed, Apr 6, 2016 at 7:43 PM, sbryfcz notifications@github.com wrote:

Any status on this bug? Hitting the same issue.

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/baudehlo/node-phantom-simple/issues/109#issuecomment-206622917

[ohenepee]

No!! What do you even understand or mean by THROTTLING? That photo above is from a 100Mbps link VPS (downloads speeds maxes past 10MB/s)... so no it has nothing to do with throttling!

[baudehlo]

I mean Facebook throttling you for hitting the page over and over.

On Apr 8, 2016, at 2:30 AM, ohenepee notifications@github.com wrote:

No!! What do you even understand or mean by THROTTLING? That photo above is from a 100Mbps link VPS (downloads speeds maxes past 10MB/s)... so no it has nothing to do with throttling!

— You are receiving this because you commented. Reply to this email directly or view it on GitHub

[sbryfcz]

Regardless, I wouldnt expect Phantom to crash, just as I wouldnt expect a browser to crash. But I'm admittedly new to Phantom. Maybe I'm missing a better practice.

[tsirolnik]

The bug is still happening - Happened to me while using node-horseman

[ohenepee]

After a few production runs, it turns out that Phantom actually crashes as a result of CPU max'ing... when it HITS 100% CPU usage and still needs more CPU power to do what it wants to do... simple solution; get a faster computer and prove me wrong.

It might not just be CPU resource on your end... RAM can also contribute

[8acker]

+1

[hibearpanda]

+1

Getting the same issue, instance just stops after hitting this error according to my logs

[ohenepee]

This is an issue with PhantomJS itself.. so you guys will have to base with this annoying bug. You can simply save state and repeat the activity as a workaround