Definitive solution for Lock error not acquired/owned by you

[Wizzzz]

@CheshireCaat

Hello, I would like to open a new issue to know if there is an example on the Lock error when you are in multi-thraeds, I have tried everything and I can not make sure not to have the error.

I'm attaching my code but I really don't see how it's possible that I still get the error.

To put it simply, in my current code I've generated a brand new engine in a folder, and as soon as I create a browser, I copy the engine and use it with setWorkingFolder

When I run just one task (say a scrapper on site A), I don't have any problems, but if I run my second scrapper on site B, after a few minutes I'll get the error.

Is there really a way to launch several scrappers with the package?

I can't hide the fact that I've been working full-time on this for 4 days now and I'm really starting to lose patience because it's putting me behind on all my projects :(

I can pay for durable solution if someone have

` import { FetchOptions } from 'browser-with-fingerprints'; import crypto from 'crypto'; import fs from 'fs'; import { createCursor } from 'ghost-cursor-playwright'; import { homedir } from 'os'; import path from 'path'; import { BrowserContext } from 'playwright-core'; import { plugin } from 'playwright-with-fingerprints';

import { Logger } from '../../logger'; import { Proxy } from '../../proxy'; import { BrowserOptions } from '../../types/browser'; import { capsolverDefaultConfig, CapsolverOptions } from '../../types/capsolver'; import { nopechaDefaultConfig, NopechaOptions } from '../../types/nopecha'; import { sleep } from '../../utils'; import { humanClick, humaneMove, humanType } from '../humanize';

type InstanceOptions = { browserOptions: BrowserOptions; capsolverOptions?: CapsolverOptions; nopechaOptions?: NopechaOptions; proxy?: Proxy; taskDuration?: number; };

type PageOptions = { useHumanization?: boolean; blockImageDomains?: string[]; };

export class BrowserInstance { private browser: BrowserContext | null = null; readonly instanceOptions: InstanceOptions;

constructor(instanceOptions: InstanceOptions) {
    this.instanceOptions = instanceOptions;

    this.instanceOptions.taskDuration = this.instanceOptions.taskDuration || 10;

    if (this.instanceOptions.capsolverOptions) {
        const configPath = path.join(__dirname, '..\\..\\extensions\\capsolver\\assets\\config.json');
        const config = { ...capsolverDefaultConfig, ...this.instanceOptions.capsolverOptions };
        fs.writeFileSync(configPath, JSON.stringify(config));
    } else if (this.instanceOptions.nopechaOptions) {
        const manifestPath = path.join(__dirname, '..\\..\\extensions\\nopecha\\manifest.json');
        const config = { ...nopechaDefaultConfig, ...this.instanceOptions.nopechaOptions };
        const manifest = JSON.parse(fs.readFileSync(manifestPath, 'utf8'));
        manifest.nopecha = config;
        fs.writeFileSync(manifestPath, JSON.stringify(manifest));
    }

    // ignore unhandled playwright-with-fingerprint exceptions
    if ((global as any).ignoreUncaughtException !== true) {
        Logger.error('Uncaught exceptions are not being handled');
        (global as any).ignoreUncaughtException = true;
        (global as any).browserError = 0;
        process.on('uncaughtException', err => {
            console.error('uncaughtException:', err);
            (global as any).browserError++;
            if ((global as any).browserError > 50) {
                Logger.error('Too many browser exceptions, exiting process');
                process.exit(1);
            }
        });
    }
}

private async startBrowser() {
    // set extensions path
    const capsolverPath = path.join(__dirname, '..\\..\\extensions\\capsolver');
    const nopechaPath = path.join(__dirname, '..\\..\\extensions\\nopecha');

    // generate workdir
    if (!this.instanceOptions.browserOptions.defaultWorkerPath)
        this.instanceOptions.browserOptions.defaultWorkerPath = path.join(homedir(), 'Desktop', 'worker');

    if (!this.instanceOptions.browserOptions.workDir) {
        const enginesPath = path.join(__dirname, '..', '..', '..', '..', '..', 'engines');
        console.log(enginesPath);
        if (!fs.existsSync(enginesPath)) fs.mkdirSync(enginesPath, { recursive: true });

        const uuid = crypto.randomUUID();
        this.instanceOptions.browserOptions.workDir = path.join(enginesPath, uuid);
        fs.mkdirSync(this.instanceOptions.browserOptions.workDir);
    }

    // copy clean worker to workdir
    this.copyDirectory(
        this.instanceOptions.browserOptions.defaultWorkerPath,
        this.instanceOptions.browserOptions.workDir
    );

    plugin.setWorkingFolder(this.instanceOptions.browserOptions.workDir);

    // set tags
    const device: FetchOptions = {
        tags: this.instanceOptions.browserOptions.tags || ['Microsoft Windows', 'Chrome']
    };

    // if chrome device, get and set last version (use min/max browser version bc impossible use useBrowserVersion)
    if (device.tags && device.tags.includes('Chrome')) {
        const versions = await plugin.versions('extended');
        const browserVersion = versions[0]['browser_version'];
        const reducedBrowserVersion = parseInt(browserVersion.split('.')[0]);

        device.minBrowserVersion = reducedBrowserVersion;
        device.maxBrowserVersion = reducedBrowserVersion;
    }

    // if profile set, use it
    if (
        this.instanceOptions.browserOptions.profilePath &&
        fs.existsSync(this.instanceOptions.browserOptions.profilePath)
    ) {
        plugin.useProfile(this.instanceOptions.browserOptions.profilePath, {
            loadFingerprint: true,
            loadProxy: true
        });
    }

    // fetch fingerprint
    const fingerprint = await plugin.fetch(this.instanceOptions.browserOptions.fingerprintSwitcherKey, device);
    const args = this.instanceOptions.browserOptions.args || [];
    plugin.useFingerprint(fingerprint);

    // loads captcha solvers extensions
    if (this.instanceOptions.capsolverOptions) args.push(`--load-extension=${capsolverPath}`);
    else if (this.instanceOptions.nopechaOptions) args.push(`--load-extension=${nopechaPath}`);

    // set proxy
    if (this.instanceOptions.proxy)
        plugin.useProxy(this.instanceOptions.proxy.fullAddress(), {
            detectExternalIP: true,
            changeGeolocation: true,
            changeWebRTC: true,
            changeTimezone: true,
            changeBrowserLanguage: true
        });

    // launch browser
    try {
        let isLaunch = false;
        await Promise.race([
            sleep(30000).then(() => {
                if (!isLaunch) throw '[BROWSER INSTANCE] Timeout during launch';
            }),
            new Promise((resolve, reject) => {
                plugin
                    .launchPersistentContext(this.instanceOptions.browserOptions.profilePath, {
                        headless: this.instanceOptions.browserOptions.headless,
                        args
                    })
                    .then((browser: BrowserContext) => {
                        isLaunch = true;
                        this.browser = browser;
                        resolve(browser);
                    })
                    .catch((error: any) => {
                        reject(error);
                        throw error;
                    });
            })
        ]);
    } catch (error) {
        (global as any).browserError++;
        if ((global as any).browserError > 50) {
            Logger.error('Too many browser exceptions, exiting process');
            process.exit(1);
        }
        throw error;
    }
}

public async init() {
    if (this.browser) throw 'Browser is already initialized';

    // blocks instance creation if one is already in progress
    let isPossibleToCreateBrowser = false;
    if (!fs.existsSync(`${process.env.TMP as string}\\lastBrowserGeneration.txt`))
        fs.writeFileSync(`${process.env.TMP as string}\\lastBrowserGeneration.txt`, '0');

    do {
        const lastBrowserGeneration = parseInt(
            fs.readFileSync(`${process.env.TMP as string}\\lastBrowserGeneration.txt`, 'utf8')
        );
        if (Date.now() - lastBrowserGeneration > 90000) {
            fs.writeFileSync(`${process.env.TMP as string}\\lastBrowserGeneration.txt`, Date.now().toString());
            isPossibleToCreateBrowser = true;
        } else await sleep(1000);
    } while (!isPossibleToCreateBrowser);

    try {
        const isBrowserCreated = await Promise.race([
            sleep(90000).then(() => false),
            this.startBrowser()
                .then(() => true)
                .catch(async error => {
                    console.error(error);
                    await sleep(90000);
                    return false;
                })
        ]);

        if (!isBrowserCreated) throw '[BROWSER INSTANCE] Timeout or error during browser creation';

        if (!this.browser) throw '[BROWSER INSTANCE] Browser failed to initialize';
    } catch (error) {
        if (this.browser) await this.close();
        throw error;
    } finally {
        await sleep(10000);
        fs.writeFileSync(`${process.env.TMP as string}\\lastBrowserGeneration.txt`, '0');
    }
}

public async newPage(options: PageOptions = {}) {
    if (!this.browser) throw '[BROWSER INSTANCE] Browser is not initialized';

    const page = await this.browser.newPage();
    if (options.useHumanization) {
        // @ts-ignore
        page.humaneMove = humaneMove;
        // @ts-ignore
        page.humanClick = humanClick;
        // @ts-ignore
        page.humanType = humanType;
        // @ts-ignore
        page.cursor = await createCursor(page);
    }

    if (options.blockImageDomains && options.blockImageDomains.length > 0) {
        await page.route('**/*', async route => {
            const request = route.request();

            if (request.resourceType() === 'image') {
                const url = request.url();
                for (const domain of options.blockImageDomains!) if (url.includes(domain)) return await route.abort();
            }

            return await route.continue();
        });
    }

    return page;
}

public async close() {
    if (!this.browser) throw '[BROWSER INSTANCE] Browser is not initialized';
    await this.browser.close();
}

public getBrowser() {
    return this.browser;
}

private copyDirectory(src: string, dest: string) {
    if (!fs.existsSync(dest)) fs.mkdirSync(dest, { recursive: true });

    const items = fs.readdirSync(src, { withFileTypes: true });

    items.forEach(item => {
        const srcPath = path.join(src, item.name);
        const destPath = path.join(dest, item.name);

        if (item.isDirectory()) this.copyDirectory(srcPath, destPath);
        else fs.copyFileSync(srcPath, destPath);
    });
}

}

`

[CheshireCaat]

I was able to run two browsers within the same process using most of your code, but with some caveats:

import { BrowserInstance } from './browser';

async function main() {
  const commonOptions = {
    headless: false,
    creationTimeout: 90000 * 10,
    fingerprintSwitcherKey: 'FINGERPRINT_KEY',
  };

  const instance1 = new BrowserInstance({
    browserOptions: {
      ...commonOptions,
      profilePath: './testProfile1',
    },
  });

  const instance2 = new BrowserInstance({
    browserOptions: {
      ...commonOptions,
      profilePath: './testProfile2',
    },
  });

  await Promise.allSettled([instance1.init(), instance2.init()]);
}

main();

1. First of all, you need to increase the timeout for launching the browser - for the entire method and for the launch itself. Where you have code whose result is saved in isBrowserCreated, you need to significantly increase it, because the engine may take much longer to download and decompress completely, otherwise errors are possible.
2. The same applies to the timeout in the section with the launchPersistentContext usage, where the timeout was doubled.
3. I have encountered a rare error that I have not seen before, related to the bas-remote-node package - I will definitely fix it in future updates, while you can patch it yourself. Right before this line, you need to add the following:

if (!fs.existsSync(this._scriptDir)) return;

With this in mind, your code worked for me without errors and launched two browsers, although I had to remove the import of some types and the use of extensions. If all this does not fix the problem, please send me the complete project with all types and data to repeat by email cheshirecat902@gmail.com. Or you can make a private repository and invite me, as it will be more convenient for you - I will look at it in my free time and try to identify the problem if it remains.

[Wizzzz]

The problem is not at launch after several minutes, I would make a repo during the day that stimulates tasks to show my problem, thank for response

[dr3adx]

The problem is not at launch after several minutes, I would make a repo during the day that stimulates tasks to show my problem, thank for response

Hey bro, add me on discord -- so we can properly discuss this

[Wizzzz]

@CheshireCaat I've just thought about it, in view of the way I manage engines, wouldn't it be possible in my case to modify your code so that it doesn't lock the file?

Added @dr3adx

[CheshireCaat]

@Wizzzz you can try, at least for the bas-remote and for the browser-with-fingerprints. But it can lead to other issues, so in such case i can't help.

[Wizzzz]

@CheshireCaat okok I won't hide the fact that this is my last solution, as I would have tried everything else.

I invited you to my repo where I manage my browser and sent an email with details, if you have time to look at my email which is very serious!

[Wizzzz]

Hi, I'm back to give you an update on my last message.

Since I deactivated the lock on the package, I have no more lock problems. It's been 4 days now that I have no problems.

Here's the link to the modified package for those interested. You'll have to import it with npm using the github link, as I haven't posted it on npm. https://github.com/Wizzzz/playwright-with-fingerprints-without-lock

If you want to reproduce my code, you have 2 choices, either download FastExecuteScriptProtected.x64, put it on your desktop and build a folder with your node in the same way as the package, all you have to do is 'setWorkingFolder' and the package will take care of the rest.

The second, faster option is to first extract FastExecuteScriptProtected.zip and build the folder yourself, then copy the folder for each instance you make.

You'll also need to create a powershell script that automatically deletes the folders you've created, because in less than 2 hours you're likely to run out of space on your machine (in my case, I've got 1to).

@CheshireCaat On the other hand, I'm facing a problem I've had before, and I don't really know how to solve it. Sometimes, when I launch the browser, it never launches, and what's more, it completely blocks my node, making it impossible to continue except by shutting it down and relaunching manually. I do use 'setRequestTimeout', though, and I've even made a home-made timeout with promise.race await Promise.race([ sleep(MAX_LAUNCH_TIMEOUT).then(() => { if (!isLaunch) throw '[BROWSER INSTANCE] Timeout during launch'; }), new Promise((resolve, reject) => { plugin .launchPersistentContext(this.instanceOptions.browserOptions.profilePath, { headless: this.instanceOptions.browserOptions.headless, args }) .then((browser: BrowserContext) => { isLaunch = true; this.browser = browser; resolve(browser); }) .catch((error: any) => { reject(error); throw error; }); })

If anyone has an idea for solving this problem, I'd be very happy to hear it, as it's currently my last problem with the package

[Wizzzz]

@CheshireCaat Hi, sorry for the message, but I'd like to ask you again about the problem with the browser that never launches. Unfortunately it's totally blocking my node and I have to restart it manually.

This is my latest problem with the package

[CheshireCaat]

@Wizzzz hi, please check your email, I replied to your message there.