Cuadrix / puppeteer-page-proxy

Additional module to use with 'puppeteer' for setting proxies per page basis.
428 stars 100 forks source link

Can't login on netflix with proxy #18

Closed mckraemer closed 4 years ago

mckraemer commented 4 years ago

I am trying to use this module with a NordVPN socks5 proxy. I can visit the login page, however when I login I get the error: (failed)net::ERR_FAILED When I use the normal VPN client it works without any issues.

const puppeteer = require('puppeteer');
const useProxy = require('puppeteer-page-proxy');

const browser = await puppeteer.launch({
        headless: config.headless
    })  
const page = await browser.newPage();
await useProxy(page, 'socks5://${vpnusername}:${vpnpassword}@de550.nordvpn.com:1080')

await page.goto('https://www.netflix.com/en/Login')
haz3haz3 commented 4 years ago

lol got the same problem right now

sometimes it works and sometimes not same error

maybe related to nordvpn this company sucks hard lately.. websites just loading endless and never finish or I get like you net::ERR_FAILED directly with every socks

when I visit the same website in other tab manually (where socks is not included) the website instant loading.. not sure if the problem is caused by nordvpn or this module here


Edit: I got a old browser profile from a few days where it does work but when iam on a specific side I always get: (node:15050) UnhandledPromiseRejectionWarning: TypeError: Cannot read property '_id' of null at Page.$puppeteerPageProxyHandler (/node_modules/puppeteer-page-proxy/src/core/proxy.js:10:31)

and the website doesnt finish loading.. the page itself is fully visible but the loading screen in tab never stops and the error comes over again and again

Cuadrix commented 4 years ago

If I am right, NordVPN's username is actually an email which contains an '@' symbol. It could be that it's causing the parser trouble. I will check it out

haz3haz3 commented 4 years ago

I deleted the userDataDir arg that I get always a fresh guest profile and with socks its like described about that 99% I get net::ERR_FAILED with any socks5. However sometimes it works but then we website just never stop loading..

when I use https proxies I get directly on any page: (node:16755) UnhandledPromiseRejectionWarning: RequestError: Socket is closed at ClientRequest. (node_modules/got/dist/source/core/index.js:788:25) at Object.onceWrapper (events.js:422:26) at ClientRequest.emit (events.js:327:22) at ClientRequest.origin.emit (node_modules/@szmarczak/http-timer/dist/source/index.js:39:20) at Socket.socketErrorListener (_http_client.js:467:9) at Socket.emit (events.js:315:20) at emitErrorNT (internal/streams/destroy.js:100:8) at errorOrDestroy (internal/streams/destroy.js:158:7) at onwriteError (_stream_writable.js:391:3) at processTicksAndRejections (internal/process/task_queues.js:86:21) at Socket._writeGeneric (net.js:772:8) at Socket._write (net.js:794:8) at doWrite (_stream_writable.js:377:12) at clearBuffer (_stream_writable.js:517:7) at Socket.Writable.uncork (_stream_writable.js:317:7) at ClientRequest._flushOutput (_http_outgoing.js:883:10) at ClientRequest._flush (_http_outgoing.js:852:22) at _http_client.js:323:47 at callSocketMethod (_http_client.js:818:7) at ClientRequest.onSocket (_http_client.js:823:7)

no nordvpn username is just a username. and your module worked fine before a few days. It suddenly stopped working with those weird issues

Cuadrix commented 4 years ago

I will test it out with some more proxies and try to pinpoint the problem

Cuadrix commented 4 years ago

I deleted the userDataDir arg that I get always a fresh guest profile and with socks its like described about that 99% I get net::ERR_FAILED with any socks5. However sometimes it works but then we website just never stop loading..

when I use https proxies I get directly on any page: .......

no nordvpn username is just a username. and your module worked fine before a few days. It suddenly stopped working with those weird issues

Are you using any authentication with those https proxies that you tested?

haz3haz3 commented 4 years ago

Yes https and socks5 are from nordvpn

Cuadrix commented 4 years ago

What chrome version are you using? I'm interested in the errors above^^

haz3haz3 commented 4 years ago

I use chromium v83 (Version 83.0.4103.61 I use on my local chromium )

puppeteer is just freshly downloaded the latest version I also tried my local installed chromium as executablePath: '/snap/bin/chromium'

which reproduces the same error

I used your module literally before 3-4 days without any problems, and your last git update was before 7 days so dont really think your module is the problem. Did chromium update the last days from v82 to v83 maybe?

I dont have any other proxies with auth so I can´t test other proxies to check if the problem is caused by nordvpn

hope we can fix this your module is just awesome and hope we can use it soon as possible again :)

Cuadrix commented 4 years ago

It's strange because someone who uses this module in their project just recently told me that their socks5 proxy worked with authentication.

From my testing, it seems like NordVPN http proxies with auth work just fine. This might just be NordVPN having wrong labels on the proxy servers, but you said that https and socks5 proxies from them worked fine a few days ago...? hmmm

haz3haz3 commented 4 years ago

The problem seems to be chromium related, cause puppeteer and local chromium on linux ubuntu cause the errors as described above while with Google Chrome instead it works (Version 83.0.4103.61)

executablePath: '/usr/bin/google-chrome'

and yes before 3-4 days it worked fine with chromium, maybe they released a new version which causes those bugs

haz3haz3 commented 4 years ago

Nevermind forget what I wrote, It worked at the start but now it comes again even with google chrome.. :(

Cuadrix commented 4 years ago

I just tested with another proxy provider, and their socks5 proxy worked perfectly with authentication. This must be NordVPN doing a sloppy job labeling what protocol their proxies are

Cuadrix commented 4 years ago

I'm still not sure whether your errors are related to this though. Could you test this proxy for me?; socks5://wpeejmlo-4:y2e001ppobvx@192.186.154.202:1080 Should be a UK proxy

haz3haz3 commented 4 years ago

Yes this company sucks hard.. The linux app is so bugged not even funny anymore.. and the chrome extension didnt worked for 2 years straight..

however not sure what to do now I get it with chromium and google chrome on fresh profiles and fresh installed browsers

what does this RequestError: Socket is closed mean? and why does it only comes with https and not socks?

literally no https proxy of nordvpn is working and all give this socket closed error. This is a example of proxy: { "id": 946987, "ip_address": "89.187.173.140", "search_keywords": [ "sha512", "P2P" ], "categories": [ { "name": "Standard VPN servers" }, { "name": "P2P" } ], "name": "United States #4425", "domain": "us4425.nordvpn.com", "price": 0, "flag": "US", "country": "United States", "location": { "lat": 25.7738889, "long": -80.1938889 }, "load": 26, "features": { "ikev2": true, "openvpn_udp": true, "openvpn_tcp": true, "socks": false, "proxy": true, "pptp": false, "l2tp": false, "openvpn_xor_udp": false, "openvpn_xor_tcp": false, "proxy_cybersec": true, "proxy_ssl": true, "proxy_ssl_cybersec": true, "ikev2_v6": false, "openvpn_udp_v6": false, "openvpn_tcp_v6": false, "wireguard_udp": true, "openvpn_udp_tls_crypt": false, "openvpn_tcp_tls_crypt": false, "openvpn_dedicated_udp": false, "openvpn_dedicated_tcp": false, "skylark": false } }


Yes I will try it give me one moment

haz3haz3 commented 4 years ago

Yes with your socks its working :((( but for me it was isreal not uk

when you got https proxy I cant test aswell to see if the socket error comes there too

Cuadrix commented 4 years ago

What about: socks5://wpeejmlo-dest:y2e001ppobvx@193.8.94.225:1080 Should be Germany

Cuadrix commented 4 years ago

There is no instructions on how to setup https proxies in NordVPN's website. I only found this: https://support.nordvpn.com/Connectivity/Proxy/1047410682/Proxy-setup-on-Chrome.htm Which instructs on how to setup http proxy on chrome This could mean that NordVPN doesn't even support https proxies in the first place -

That socket error earlier can happen when the request suddenly aborts due to some issues connecting. Usually those issues are related to the port or protocol. Since NordVPN uses 80 (default http port) as their port number for http proxies, then maybe they also use the default port for https, which is 443 ?

So maybe try something like https://user:pass@nordvpn-host:443

haz3haz3 commented 4 years ago

No both are same ip for me in isreal

current IP: 193.8.94.225 current Country of IP: Israel (IL)

however its working so clearly nordvpn is the problem :(

before 3-4 days I used socks only so cant say if https worked before too. I used tested https proxies today cause of the errors

Yes I always used the socks and proxies like this socks= socks5://user:pass@server:1080 https= https://user:pass@server:89 http= http://user:pass@server:80

Cuadrix commented 4 years ago

I tested the NordVPN proxies again, and they worked. http, https, and socks5 all worked

haz3haz3 commented 4 years ago

Maybe its my system not sure. Now nordvpn socks works again.

I´am really sorry for maybe wasting your time. I will try to install a fresh ubuntu and try again, maybe my system corrupted. however the person who started the thread here with netflix had the same error so I thought its not my system or code

however realy really thank you for try helping me really appreciate that. I will come back later when I did more testing on new operating system

Cuadrix commented 4 years ago

Ok. just start a new issue if it persists ^^ Closing this for now

haz3haz3 commented 4 years ago

I wanted to add to this thread in case that other people came to the same problem that nordvpn has limitation of proxy rotating.

they sayed their terms against web scrapping and their server block requests of proxy/socks rotating or general bot stuf.. (no joke shit company) I found out that when I use 1x proxy solo and dont change it, then it works

also their have a max device limit of 6 devices. And proxy/socks usage counts aswell as 1x device. so if you reach the limit you may aswell cant connect

so puppeteer-page proxy is cleary not the problem for this thread here. Sorry again for thinking the problem was caused by this amazin module :) greets

Dam998 commented 4 years ago

I want to add more information about this error I have the same problem, I am using blankproxies.co.uk residential proxies I solved it because I noticed this problem with some but not all proxies, so I removed the "sick" proxies from my software and now everything works fine