ulixee / hero

The web browser built for scraping
MIT License
649 stars 32 forks source link

Mitm causing max of 4 heros on windows pc with "cheap" wifi adapters (edited title) #220

Open shapebuster opened 1 year ago

shapebuster commented 1 year ago

I am having some issues trying to run my program on a dell optiplex with a usb wifi adapter. My program just hangs randomly. Scraping in a loop just stops and other hero websites never load (infinite loading circle). Comparatively, my lenovo s540 with built in wifi card never has this issue. Same with my macbook pro 2015. Smooth as butter. I'm just wondering if an external wifi card might be the reason?

ulixee 2.0.0-13

Pc that fails: Dell optiplex 16gb ram i5-8500 Usb wifi adapter

Pc that succeeds: Lenovo S540 16gb ram AMD Ryzen 4000 series Macbook Pro 2015 16gb ram i7 processor

My program runs smoothly on the lenovo and macbook. When the program fails on the PCs it doesn't tell me anything, its like everything just freezes and then theres some network errors that pop up in dev tools on each hero instance.

I have tried the following and still no luck:

blakebyrnes commented 1 year ago

Could you try disabling the mitm in hero constructors?

blakebyrnes commented 1 year ago

Update from discord: this seems to relieve issues when running with disableMitm: true. @paulosllvn to try a plugin that disables tcp settings to see if that's the issue.

blakebyrnes commented 1 year ago

Update: TCP disabled not the issue. Still maxes out at 4 concurrent Heros on those machines.