rubycdp / ferrum

Headless Chrome Ruby API
https://ferrum.rubycdp.com
MIT License
1.71k stars 123 forks source link

Passes proxy settings to remote Chrome #275

Closed zedtux closed 1 year ago

zedtux commented 2 years ago

This commit passes the 'proxy-server' parameter to a remote Chrome instance (meaning passing a :url parameter) so that you can use something like the puffing-billy gem as a MITM proxy to mock requests.

In my case, I have a dockerized Ruby On Rails application, using the Cucumber testing framework, with capybara and cuprite in order to manipulate a Chrome instance running in a separated container. My app is using the Paddle payment processor and we'd like to mock calls to their APIs so we're using the puffing-billy gem which fires a proxy server acting as a VCR to record/replay requests.

Being in the process of implementing the Cuprite driver to puffing-billy, we were unable to get Chrome using the puffing-billy server until the change from this PR.

zedtux commented 2 years ago

Actually, I'm not sure this PR make sense : my remote browser is in a container which I have the control, so I can add the flags myself.

I had websocket errors during my test, running this PR. I wanted the proxy to bypass websocket requests. I found from the dev console I can do it, like I did in this PR to set the proxy server, but to do it quickly, I've updated the chrome container's arguments in order to pass the --proxy-bypass-list=ws://* flag. That didn't work until I stopped using this PR and passing manually both flags --proxy-server and --proxy-bypass-list to the chrome container.

At that moment I realised how useless is my PR in my case.

Therefore this PR could make sense in the case :

Anyone can give input in this PR please ?

route commented 1 year ago

I think in general it doesn't make sense to pass proxy options, passed to the Browser#new, to #create_page as they end up as CLI flags for Chrome and affect the whole browser anyway. So these are in different words CLI flags.

I've introduced proxy options in https://github.com/rubycdp/ferrum/pull/300 so that you can call create_page(proxy: { host: "host", port: "port", user: "user", password: "password", bypass: "" })

Does it solve the issue?

zedtux commented 1 year ago

affect the whole browser anyway

Well, in my use case it should affect the whole browser.

I've introduced proxy options in https://github.com/rubycdp/ferrum/pull/300 so that you can call create_page(proxy: { host: "host", port: "port", user: "user", password: "password", bypass: "" })

Does it solve the issue?

Well, it should then be supported by the cuprite gem so I guess not (yet) but I haven't checked.

Anyway, so far I solved my issue by updating the Chrome docker image environment so that it forces the proxy for the whole browser and my test suite now passes mocking external calls (VCR), so if you want to close that PR it's totally fine for me.

route commented 1 year ago

I think this is a good solution for your specific case. I'm not sure how to support it in Cuprite because session usually created automatically, so there's no way of passing proxy options when creating a page.

n1xn commented 1 year ago

Actually url is not combinable with proxy and will result in not setting the proxy accordingly. I had to evade by creating the browser only with url option and applying the proxy to the create_page method. This took me a lot of time to figure out, I hope this helps somebody out.

# No
Browser.new(url: 'http://localhost:4000', proxy: { host: 'proxy_host', port: 'proxy_port', user: 'proxy_user', password: 'proxy_password'})
# Yes
browser = Browser.new(url: 'http://localhost:4000')
page = browser.create_page(proxy: { host: 'proxy_host', port: 'proxy_port', user: 'proxy_user', password: 'proxy_password'})
page.go_to('https://whole-day-gone.fml.truth')