joelgriffith / navalia

A bullet-proof, fast, and reliable headless browser API
https://joelgriffith.github.io/navalia/
GNU General Public License v3.0
957 stars 33 forks source link

Support for authenticated proxies #37

Open andrewlinfoot opened 7 years ago

andrewlinfoot commented 7 years ago

Love this repo! I'm thinking about switching over one of my projects from NightmareJS to this but I need to use authenticated proxies. I know with the chrome command line flag you can pass in a proxy via --proxy-server=PROXY_URL_HERE. However, this only works for non-authenticated proxies. In NightmareJS they handle this by listening for an authentication event and then logging in.

How could I implement something similar for this repo?

If you can provide me some guidance, I'd be willing to implement the PR.

joelgriffith commented 7 years ago

Thanks for requesting this! I haven't dug too far into chrome's proxy functionality (especially when authentication is concerned). I've seen some discussion about it elsewhere, so I'll have to do some archaeology and get back to you. Thanks for the link as well, that will likely come in handy for us!

joelgriffith commented 7 years ago

Sweet, so I think I have this working in a branch if you're interested in testing 🙃 0.2.3-0 has just been published and supports two things: proxies and basic-auth. To use both do:

const { Chrome } = require('navalia');
const chrome = new Chrome({
  flags: {
    proxyServer: 'http://my-cool-proxy.com'
  }
});

chrome
  .auth('username', 'password') <-- fills in auth details when the proxy prompts
  .goto('http://www.google.com')
  .then((res) => console.log(res))

If this works I'll do a stable release and document it. I'm naively assuming your proxy (and most) do basic-auth, which could be mistaken (but appears to be how electron/nightmare handle it).

Let me know!

andrewlinfoot commented 7 years ago

Awesome, thanks! Yep, pretty sure most use basic auth. Will check it out asap in the morning :)

On Tue, Jul 25, 2017, 10:18 PM Joel Griffith notifications@github.com wrote:

Sweet, so I think I have this working in a branch if you're interested in testing 🙃 0.2.3-0 has just been published and supports two things: proxies and basic-auth. To use both do:

const { Chrome } = require('navalia'); const chrome = new Chrome({ flags: { proxy: 'http://my-cool-proxy.com' } });

chrome .auth('username', 'password') <-- fills in auth details when the proxy prompts .goto('http://www.google.com') .then((res) => console.log(res))

If this works I'll do a stable release and document it. I'm naively assuming your proxy (and most) do basic-auth, which could be mistaken (but appears to be how electron/nightmare handle it).

Let me know!

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/joelgriffith/navalia/issues/37#issuecomment-317950410, or mute the thread https://github.com/notifications/unsubscribe-auth/ABPr_c6pfG2srDnFjVnYP_igyI4-OSjeks5sRsxAgaJpZM4Oht5S .

andrewlinfoot commented 7 years ago

Am working on testing this but I can't get it to work. My cdp.Network.requestIntercepted callback is never called. Any thoughts? Currently running on the latest version of chrome canary v62.0.3167.0 on Mac.

Btw the request interception methods weren't implemented until chrome v61. The current stable release is v60. We might need to add some documentation so people know they need to install canary for this feature.

joelgriffith commented 7 years ago

Could you perhaps share your script? It's possible that there's a race condition internally... I can work on getting some "best practices" docs together for Canary. It's also possible that some proxies don't do auth in the way that this protocol expects... I can dig some later on this

joelgriffith commented 7 years ago

Also this just launched: https://github.com/graphcool/chromeless and has gained a lot of traction already. It's likely I'll be closing this down in favor of their repo

andrewlinfoot commented 7 years ago

I literally just got the email notification from one of my Github issues subscriptions and was reading the docs. I'll probably go with their implementation as well. Thanks for all the work you did on Navalia though! :)

joelgriffith commented 7 years ago

I'm not going to give up just yet. If you're still interested, let me know and we can work through it

andrewlinfoot commented 7 years ago

@joelgriffith curious why you still want to work on Navalia vs contributing to Chromeless? I don't know much how Chromeless or Navalia work under the hood but the APIs seem almost identical and Chromeless seems to have a lot more community support. Also the Chromeless AWS Lambda integration is super awesome.

Chromeless definitely seems less feature complete and has worse docs :( Hoping that will work itself out pretty quick though.

joelgriffith commented 7 years ago

I have some very good news in this regard, and will be sharing more broadly once the details settle 😉

AshCoolman commented 6 years ago

Any update on this news :)?

Navalia is my favourite framework - the Dev exp totally blows me away. But still, I cannot pass up the Chromeless promise of taking ~20+ minutes of e2e tests, and distributing them across mere seconds.