Failed to load in the timeout specified

[AWebOfBrown]

When trying to use Navalia, I don't seem to be able to connect to chrome as any operation simply timesout. At first I thought this was an issue with my chrome install, but I can connect to it if I install and use chrome-launcher seperately. I'm using ubuntu 16 with version 60 of google-chrome-stable.

Any ideas what might be going wrong? Edit: I can get Selenium working with chrome so I don't think it's anything to do with the chrome install.

[joelgriffith]

There's a few ways to go about debugging this. Unfortunately Chrome on Linux is a bit unstable, but is coming along. You'll also want the absolute latest version (Canary) as well. Here's how to get debug logs:

DEBUG=navalia:chrome node my-script

This will have Navalia print verbose debug info.

You can also try sandbox mode to see if that helps. I've heard elsewhere that sand boxing can fix some issues:

const chrome = new Chrome({
  flags: {
    noSandbox: true,
    headless: true,
    disableGpu: true,
  }
});

Which might help if Chrome isn't allowing remote connections

[joelgriffith]

Also just added an FAQ. Would be more than happy to capture your issue there once we find it. https://github.com/joelgriffith/navalia/blob/master/FAQ.md

[AWebOfBrown]

Thanks for the help Joel. As far as I'm aware, Google won't support canary on Linux as it requires too much work to do nightly packages. I'll try the debugging route, thanks!

[AWebOfBrown]

The debug log reads:

  navalia:chrome :getChromeCDP() > starting chrome +5ms

  navalia:chrome :WARN > Retrying 1 time(s) due to issue: 'Error: connect ECONNREFUSED 127.0.0.1:40937' +7s

  navalia:chrome :getChromeCDP() > starting chrome +0ms

  navalia:chrome :getChromeCDP() > chrome launched on port 35181 +7s

  navalia:chrome :goto() > going to https://amazon.com +0ms

(node:115427) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Goto failed to load in the timeout specified

(node:115427) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

  navalia:chrome :goto() > waiting for pageload on https://amazon.com +2s

  navalia:chrome :evaluate() > executing function 'getPageURL' in Chrome +28s

This is just using the buyItOnAmazon function copy/pasted from the docs. Nothing happens past the 'getPageURL' part. I'm not sure how to tackle this as it seems Chrome is ultimately launching?

[joelgriffith]

The ECONNREFUSEF is interesting. Might try the no-sandbox flag from above. Last thing that jumps out is the lack of www in the URL (my bad). Have you tried alternative URLs or websites? I'll be exploring Linux more soon, so hopefully I can be of more use!

[AWebOfBrown]

In the previous test I was using the no-sandbox flag.

Interestingly I added in the www part of the URL, and this time didn't get the ECONNREFUSEF error..it did still fail to load amazon the first go, but then succeeded. Ultimately it failed because it couldnt find the '.buy-now' selector.

If you're curious, this is what I get logged now:

  navalia:chrome :then() > Executing 1 actions +0ms
  navalia:chrome :getChromeCDP() > starting chrome +4ms
  navalia:chrome :getChromeCDP() > chrome launched on port 34534 +631ms
  navalia:chrome :goto() > going to https://www.amazon.com +1ms
  navalia:chrome :WARN > Retrying 1 time(s) due to issue: 'Goto failed to load in the timeout specified' +1s
  navalia:chrome :goto() > going to https://www.amazon.com +0ms
  navalia:chrome :goto() > waiting for pageload on https://www.amazon.com +40ms
  navalia:chrome :goto() > waiting for pageload on https://www.amazon.com +715ms
  navalia:chrome :evaluate() > executing function 'getPageURL' in Chrome +19s
  navalia:chrome :evaluate() > executing function 'getPageURL' in Chrome +0ms
  navalia:chrome :then() > Executing 1 actions +63ms
  navalia:chrome :wait() > waiting for selector "input" a maximum of 1000ms +1ms
  navalia:chrome :evaluate() > executing function 'waitForElement' in Chrome +0ms
  navalia:chrome :focus() > focusing 'input' +19ms
  navalia:chrome :type() > typing text 'Kindle' into 'input' +24ms
  navalia:chrome :then() > Executing 1 actions +14ms
  navalia:chrome :wait() > waiting for selector ".buy-now" a maximum of 1000ms +0ms
  navalia:chrome :evaluate() > executing function 'waitForElement' in Chrome +0ms
  navalia:chrome :WARN > Retrying 1 time(s) due to issue: 'Error: Selector ".buy-now" failed to appear in 1000 ms' +1s
  navalia:chrome :wait() > waiting for selector ".buy-now" a maximum of 1000ms +1ms
  navalia:chrome :evaluate() > executing function 'waitForElement' in Chrome +0ms

I hear amazon change their page structure often so this might be working as intended, assuming the 'goto failed to load' is simply due to the site loading slowly? Might be able to close this?

Edit: Aside from editing in the 'www' (which I should've noticed anyway), perhaps the code in the docs should be something more reliable like grabbing the site header? I don't know what the chrome connection refusal was all about though.

[AWebOfBrown]

Just a thought: many of my pageloads are taking close to 1 second to execute, as seen in the log above, the most recent I did was ~980ms and I'm getting frequent timeout errors between successes. I think Navalia has a default 1 second timeout? If so, this is what's causing a lot of my problems. The timeout is simply too quick.

[joelgriffith]

Makes sense. The docs weren't meant as a playground, so to speak. I'll go through and clean up stuff that I know doesn't work.

The timeout are configurable, so you should be able to do:

chrome.click('.some-selector', { timeout: 5000 });

I believe it's also configurable from the constructor: https://joelgriffith.github.io/navalia/chrome/constructor/#timeout

I'll go through and update doc example to highlight that they are for reference, but might not actually work (and move them to ones that do work, depending)!

[AWebOfBrown]

Ahhh I see. Sorry, I assumed that a timeout on .goto would've been taken as a second parameter, as opposed to a general async timeout set on the constructor. My apologies, think it's safe to close this now. Thanks again.

Edit: oh timeout can be set on the method? Great :)

[joelgriffith]

No problem: you should be able to do both. The constructor will set a global "default" while the the second parameter in the goto call will override it (or should). I'll try and clarify this in documentation