Closed kurohoan closed 2 years ago
Does anyone have a clue about this problem?
Maybe it doesn't work on the following modules?
puppeteer-extra https://github.com/berstend/puppeteer-extra#readme
if you want run a proxy you misunderstand the target of this module (different proxy per page). However browsers support the use of proxy in the launch arguments, that make your problem easier to solve. You can pass to the arguments to the browser launch the proxy that the browser must use, and if it necessary with the page.authentificate() you can introduce your proxy user and password. Check this example (example proxy data):
for( let i=0; i < 3; i++ ){
const browser = await puppeteer.launch({
executablePath:"C:/Program Files/Google/Chrome/Application/chrome.exe",
args: [ '--proxy-server=194.234.232.16:7777']
});
var newPage= await browser.newPage();
await newPage.authenticate({username: 'exampleUser', password: 'examplePass'});
await newPage.goto("https://www.myip.com/");
}
Thanks for the answer. But that way, I can't change the proxy while the browser is running.
I want to switch proxies without having to launch the browser multiple times.
Then my first solution is not for you.
For fix the problem of your first question (all page same ip/proxy) you need add this line:
await useProxy(page, 'http://127.0.0.1:80');
This code change the proxy of the page and you didn't use in your code, for that reason that web have the same ip (none page use a proxy ).
For change the proxy without close the browser you can reuse the same line for change the proxy in the same page (or open a new one page and use the new proxy in that new one).
I tried using useProxy(), but when three browsers are running the process at the same time, it seems that the same proxy is set in all browsers when I access the IP verification site, even though they are supposed to be set to different proxies! I would like you to check on this phenomenon.
I use the module to run multiple proxy at the same time and i didn't see that problem. I need to see the current code you are running to search the problem.
When I was modifying the code for the post, I noticed something. In fact, it uses a module called "promise.any" to return resolve when a certain condition is hit, which interrupts any other process running in Promise, but I was still returning resolve when I tested it. So I changed the code below to intentionally cause a reject. I can't run the test code below right now, so you will have to wait a bit.
const puppeteer = require('puppeteer');
const useProxy = require('puppeteer-page-proxy');
const any = require('promise.any');
const BROWSER_WORK_NUM = 3;
let promiseArray = [];
for( let i=0; i < BROWSER_WORK_NUM; i++ ){
let use_proxy = get_use_proxy( proxy_list, Number(index) + i );
promiseArray.push( check_proxy(item, pages[i], use_proxy) );
}
await any(promiseArray)
.then( res => {
console.log(res);
}).catch( e => {
console.log(e);
});
function check_proxy(item, page, use_proxy){
return new Promise( async (resolve) => {
try{
if( use_proxy ){
const proxy_fmt = 'https://'+use_proxy.user+':'+use_proxy.pass+'@'+use_proxy.host+':'+use_proxy.port;
await useProxy(page, proxy_fmt);
}
await page.goto('https://www.cman.jp/network/support/go_access.cgi', {"waitUntil":"domcontentloaded"});
throw new Error('test');
//return resolve();
}catch( e ){
console.log(e);
return reject();
}
})
}
Thanks to your help I was able to solve the problem.
The first reason is that I was using "puppeteer-extra". This has been fixed by changing to the stock puppeteer module. Also, as I mentioned in a previous comment, I was returning resolve() for testing, which caused the proxy set to not work properly.
Thanks for your help.
With the following code, I am running multiple browsers and running them concurrently in Pormise. I have different proxies set up in each browser, but when I access a site that checks IPs, the same proxy seems to be set up in all browsers.
How can I solve this problem?
` for( let i=0; i < 3; i++ ){ browser[i] = await puppeteer.launch(option); pages[i] = await browser[i].newPage();
await pages[i].setRequestInterception(true); pages[i].on('request', async (request) => { if (request.resourceType() === 'image'){ request.abort(); }else{ let proxy_fmt = 'http://user:pass@host:port'; await useProxy(request, proxy_fmt); } }); } `