search

rebrowser / rebrowser-patches

Collection of patches for puppeteer and playwright to avoid automation detection and leaks. Helps to avoid Cloudflare and DataDome CAPTCHA pages. Easy to patch/unpatch, can be enabled/disabled on demand.
https://rebrowser.net
145 stars 11 forks source link

can't use page.evaluate properly #17

Open younes-zeboudj opened 6 days ago

younes-zeboudj commented 6 days ago

Even when using 'enableDisable' approach, my code injected using page.evaluate(...) has no access to the main frame DOM. I can see console.log message done by my code but the DOM manipulations are not there. What is the problem?

nwebson commented 6 days ago

Please share your code. It should work fine.

younes-zeboudj commented 6 days ago

I will try to avoid sharing code because of some sensitive data. If we need at after all I will try to make a reproducible snippet. For now I add the following:

ProtocolError: Protocol error (Runtime.evaluate): Promise was collected

younes-zeboudj commented 6 days ago

The code is now working without changing anything... I will add new comments if the error occurs again

younes-zeboudj commented 6 days ago

The problem is still there. It seems it has nothing to do with the reported error because disabling the patch makes the code executes correctly while printing the same error

cleiltonoliveira commented 6 days ago

@younes-zeboudj Isn't this related to https://github.com/rebrowser/rebrowser-patches/issues/13?

nwebson commented 6 days ago

@younes-zeboudj I don't think I can help without seeing the code... It feels like there are some misuse of async/await + #13 in effect.

younes-zeboudj commented 6 days ago

I am not sure if it is related to #13, it may be. However, here is a minimal example:

process.env.REBROWSER_PATCHES_RUNTIME_FIX_MODE = "enableDisable";

async function main() {
  const puppeteer = require("puppeteer-core");

  const browser = await puppeteer.launch({
    headless: false,
    executablePath:
      "C:\\Program Files\\BraveSoftware\\Brave-Browser\\Application\\brave.exe",
  });

  const page = (await browser.pages())[0];
  await page.goto("https://fr.tlscontact.com/", { waitUntil: "load" });

  console.log("page loaded");

  await page.evaluate(`document.body.appendChild(document.createTextNode('test patch'))`);

  console.log((await page.content()).includes("test patch"), "url:", page.url());
}

main();

step by step:

Result

console prints:

page loaded
true url: https://fr.tlscontact.com/

In the dev tools' elements inspector you get: image

You can see that the string is not appended to the top document. Re-running the test may results in inserting the string in some other iframe. You may need to run the code a couple of times since it may by chance get inserted in the correct place.

More

Commenting the first line makes things run as expected.

younes-zeboudj commented 5 days ago

As a workaround, I am using the top window. Something like:

page.evaluate('window.top.func= ()=>{console.log('my func')}

// in the case of a complex code
page.evaluate(`window.top.setTimeout("${myJsCodeThatTargetsTopWindow}")

The downside of this approach is some complexity and risk of having the code injected in a deeper iframe in which case window.top will not refer to the main frame window.

PS: I tried puppteer's page.mainFrame().evaluate(...) with no success. Also tried 

let mainFrame= page.mainFrame()
while (mainFrame.parentFrame()) mainFrame= mainFrame.parentFrame()

mainFrame.evaluate(...)

But still the same problem is there.

nwebson commented 5 days ago

@younes-zeboudj thanks for sharing the code. It's really interesting behavior.. So, if you disable the patch (comment the first line), then this text node will be added not in the iframe, but in the parent DOM? Did I get you right?

So, I tried to run this code right in devtools, and it added the node into the root DOM as it suppopsed to add. I don't see this swagger iframe #ie-view at all on the page. Do I have to log in to be able to see it?

image

younes-zeboudj commented 5 days ago

@nwebson thanks for reporting your experimentation. Do you mean you ran the js code directly in the devtools? And did you try to run the program a couple of times? 3 times should be sufficient to get the unexpected behavior, I noticed that it gets inserted in different iframes when retrying, sometimes in the top DOM. You don't need to be logged in in order to get that result.

nwebson commented 1 day ago

@younes-zeboudj btw why do you use enableDisable approach instead of alwaysIsolated? Just noticed that.