search

rubycdp / ferrum

Headless Chrome Ruby API
https://ferrum.rubycdp.com
MIT License
1.75k stars 127 forks source link

Thread safety concerns with Proxy #209

Closed ttilberg closed 3 years ago

ttilberg commented 3 years ago

The readme currently states that Ferrum is fully threadsafe. Is this still the case with the new proxy option? Since browser.proxy_server.rotate affects global state, I'm just curious if this warrants any special handling or notes in the readme.

route commented 3 years ago

It's thread safe because both Ferrum and HTTPProxy are thread safe, the issue is when you have multiple pages they still use the same http proxy, so they can end up using the same IP with dumb webrick proxy. But with smart rotating proxy that can determine sessions and switch proxies after some time it would work much better.

route commented 3 years ago

I think I could implement smart proxy in ruby that switches proxy on every single CONNECT assuming that pages are created in a dedicated context, but this out of scope of this gem

route commented 3 years ago

And if Chrome broke the connection with the server after page close it would be even better, but it doesn't.

route commented 3 years ago

I’ll close this it’s better to open questions in discussions