ZFC-Digital / puppeteer-real-browser

This package is designed to bypass puppeteer's bot-detecting captchas such as Cloudflare. It acts like a real browser and can be managed with puppeteer.
https://www.npmjs.com/package/puppeteer-real-browser
MIT License
812 stars 96 forks source link

can not use browser.close() method properly #41

Closed ibrahimmus closed 6 months ago

ibrahimmus commented 7 months ago

I am running multiple processes in parallel. Browser.close() does not have any effect which leads to multiple chrome processes hanging.

mdervisaygan commented 7 months ago

Please add your code. If you get an error, the program stops but chrome does not. It may be caused by this.

ibrahimmus commented 7 months ago

The whole logic is in try catch and finally block, and in finally block I have await browser.close(). I will paste the code example later but FYI the same code was working with puppeteer and puppeteer stealth previously. I did not get into this situation of having multiple chrome processes hanging

ibrahimmus commented 7 months ago

Just run these two scripts together. Only run the second one, first one needs to be in the same path.

////FIRST SCRIPT

import { connect } from 'puppeteer-real-browser'

const main = async () => { const {browser, page} = await connect({ headless: 'auto', args: [], customConfig: {}, skipTarget: [], fingerprint: false, turnstile: false, connectOption: {}, tf: true, }) await page.goto('https://antoinevastel.com/bots/datadome'); await page.evaluate(() => { return new Promise(resolve => { setTimeout(resolve, 20000); }); }); await browser.close(); process.exit(0); }

main();

//SECOND SCRIPT

import { fork } from 'child_process';

const maxForks = 5; // Maximum number of forks allowed let runningForks = 0; // Initialize the number of running forks

const startFork = () => { const child = fork('./${NAME_OF_FIRST_SCRIPT}'); // Fork the main script

// Event handler for when the forked process exits
child.on('exit', () => {
    runningForks--; // Decrement the count of running forks
    if (runningForks < maxForks) {
        startFork(); // Start a new fork if the count is less than the maximum
    }
});

runningForks++; // Increment the count of running forks

}

// Start initial forks for (let i = 0; i < maxForks; i++) { startFork(); }

mdervisaygan commented 7 months ago

Hi, Puppeteer stops working when an error is received (even when closing the browser) but chromium does not. This causes the browser to hang as you mentioned. I am working on this bug and will fix it.

ibrahimmus commented 7 months ago

@zfcsoftware is there any way I could help you with this to speed it up? This is the only thing that blocks me in using/testing this in my case. Thanks

mdervisaygan commented 6 months ago

@zfcsoftware is there any way I could help you with this to speed it up? This is the only thing that blocks me in using/testing this in my case. Thanks

Hello, https://github.com/zfcsoftware/puppeteer-real-browser/blob/2627ceacd531fa599353b361f1ff0926fd544669/src/index.js#L166 You can use the chrome.close() method.

ibrahimmus commented 6 months ago

@zfcsoftware there is only chrome.kill() option. So you are telling me to get chrome instance back from connect method response and work with it as a temp solution?

mdervisaygan commented 6 months ago

The chrome variable is already returned, yes you can use it that way. Normally this problem does not occur under normal conditions, but it may occur when an error is received, I need to examine it in detail. https://github.com/zfcsoftware/cf-clearance-scraper/blob/main/module/browser.js You can create a browser like this and try that too.