kaliiiiiiiiii / undetected-playwright-python

Undetected Python version of the Playwright testing and automation library.
https://playwright.dev/python/
Apache License 2.0
115 stars 13 forks source link

[BUG] Frame Locators are failing #9

Closed Vinyzu closed 3 months ago

Vinyzu commented 6 months ago

System info

Source code

Test file (self-contained)

import asyncio
from playwright.async_api import Page as AsyncPage, async_playwright

async def bytedance():
    # playwright install chromium
    # playwright install-deps chromium
    async with async_playwright() as p:
        browser = await p.chromium.launch(headless=False)
        context = await browser.new_context(locale="en-US")
        page = await context.new_page()

        await page.goto("https://recaptcha-demo.appspot.com/recaptcha-v2-checkbox-explicit.php")
        checkbox = page.frame_locator("iframe[title='reCAPTCHA']")
        await checkbox.locator(".recaptcha-checkbox-border").click()

if __name__ == "__main__":
    asyncio.run(bytedance())

Steps

Expected Passes on default playwright

Actual

D:\System\Lib\Python\Python310\python.exe D:\System\AppData\PyCharm\scratches\scratch.py 
Traceback (most recent call last):
  File "D:\System\AppData\PyCharm\scratches\scratch.py", line 17, in <module>
    asyncio.run(bytedance())
  File "D:\System\Lib\Python\Python310\lib\asyncio\runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "D:\System\Lib\Python\Python310\lib\asyncio\base_events.py", line 649, in run_until_complete
    return future.result()
  File "D:\System\AppData\PyCharm\scratches\scratch.py", line 14, in bytedance
    await checkbox.locator(".recaptcha-checkbox-border").click()
  File "D:\Projects\PyCharm\undetected-playwright-python\undetected_playwright\async_api\_generated.py", line 15706, in click
    await self._impl_obj.click(
  File "D:\Projects\PyCharm\undetected-playwright-python\undetected_playwright\_impl\_locator.py", line 158, in click
    return await self._frame.click(self._selector, strict=True, **params)
  File "D:\Projects\PyCharm\undetected-playwright-python\undetected_playwright\_impl\_frame.py", line 494, in click
    await self._channel.send("click", locals_to_params(locals()))
  File "D:\Projects\PyCharm\undetected-playwright-python\undetected_playwright\_impl\_connection.py", line 63, in send
    return await self._connection.wrap_api_call(
  File "D:\Projects\PyCharm\undetected-playwright-python\undetected_playwright\_impl\_connection.py", line 495, in wrap_api_call
    return await cb()
  File "D:\Projects\PyCharm\undetected-playwright-python\undetected_playwright\_impl\_connection.py", line 101, in inner_send
    result = next(iter(done)).result()
undetected_playwright._impl._errors.TimeoutError: Timeout 30000ms exceeded.

Process finished with exit code 1
Vinyzu commented 6 months ago

Maight be related to https://github.com/kaliiiiiiiiii/undetected-playwright-python/issues/7 ?

exiutech commented 6 months ago

yes . I encountered the same problem .

steinerx commented 5 months ago

Try adding this flags: '--disable-web-security', '--disable-site-isolation-trials', '--disable-features=IsolateOrigins,site-per-process'

The downside of this is the cloudflare challenge will not passed anymore. I've tried using the patcher for playwright-node. It works for solving the cloudflare challenge but for some reason, accessing iframes is not working. The workaround that I've found is using the flags above. I think this is the one '--disable-site-isolation-trials' causing the cloudflare challenge to not pass but it is also the reason I can make the iframes operations working again. I'm just guessing though.

raishid commented 5 months ago

I have the same problem

luuzthebest commented 4 months ago

i have the same problem

Jaammerr commented 4 months ago

same problem :(

kaliiiiiiiiii commented 4 months ago

I'm aware of this issue, currently just don't really have the time to fix it tho. If you're relying on that, maybe using https://github.com/kaliiiiiiiiii/Selenium-Driverless is worth a consideration. It generally is more stealthy (developed from ground on to be as undetectable as possible) and receives more priority//maintainance as I don't have much hopes in this project//patch. Playwright just isn't designed to be undetectable

nandish486 commented 3 months ago

I'm aware of this issue, currently just don't really have the time to fix it tho. If you're relying on that, maybe using https://github.com/kaliiiiiiiiii/Selenium-Driverless is worth a consideration. It generally is more stealthy (developed from ground on to be as undetectable as possible) and receives more priority//maintainance as I don't have much hopes in this project//patch. Playwright just isn't designed to be undetectable

According to you, Undetectable playwright is a Patch, then it means it uses playwright module also, Then why can't You fix evaluation and script execution proxying to playwright? I guess script execution things have no match to detection?

kaliiiiiiiiii commented 3 months ago

According to you, Undetectable playwright is a Patch, then it means it uses playwright module also, Then why can't You fix evaluation and script execution proxying to playwright? I guess script execution things have no match to detection?

@nandish486 Didn't say I couldn't fix it - just don't have the time to atm. If this is urugent for you & driverless doesn't work for you, maybe consider

  1. supporting @Vinyzu and//or me (@kaliiiiiiiiii )
  2. develop//fix it yourself & submit a PR
  3. Hire someone to fix it
nandish486 commented 3 months ago

According to you, Undetectable playwright is a Patch, then it means it uses playwright module also, Then why can't You fix evaluation and script execution proxying to playwright? I guess script execution things have no match to detection?

@nandish486 Didn't say I couldn't fix it - just don't have the time to atm. If this is urugent for you & driverless doesn't work for you, maybe consider

  1. supporting @Vinyzu and//or me (@kaliiiiiiiiii )
  2. develop//fix it yourself & submit a PR
  3. Hire someone to fix it

Yes brother I would like to help you, I started reading src, I found problem, and it is javascript context.

I think your project is far more better, that's why I issued. I know there might be lot of things besides public project, for which time fades, ikik, no worries.

kaliiiiiiiiii commented 3 months ago

@nandish486

Yes brother I would like to help you, I started reading src, I found problem, and it is javascript context.

Mind giving us some some information?

  1. File(s, full path) & Line(s) reference? (if possible with url as reference)
  2. Why & where it exactly fails?
  3. What the necessary steps to fix it would be?
nandish486 commented 3 months ago

This issue has been resolved:

Upon inspecting the page source after it loads, it's apparent that the button you are attempting to click is nested within an iframe, which in turn is nested within another iframe.

iframe[name='recaptcha'] < iframe[title='reCAPTCHA'] < your button

To access the button, you'll need to use iframe.content_frame twice to navigate to the desired location.

As mentioned by @kaliiiiiiiiii in issue #12, JavaScript executions occur in isolated environments, necessitating navigation through different frames, as described above.

In short, direct access to frames within frames or to the top frame's iframe is not feasible; navigation is necessary.

This should address and resolve your problem effectively.

kaliiiiiiiiii commented 3 months ago

This issue has been resolved:

Upon inspecting the page source after it loads, it's apparent that the button you are attempting to click is nested within an iframe, which in turn is nested within another iframe.

iframe[name='recaptcha'] < iframe[title='reCAPTCHA'] < your button

To access the button, you'll need to use iframe.content_frame twice to navigate to the desired location.

As mentioned by @kaliiiiiiiiii in issue #12, JavaScript executions occur in isolated environments, necessitating navigation through different frames, as described above.

In short, direct access to frames within frames or to the top frame's iframe is not feasible; navigation is necessary.

This should address and resolve your problem effectively.

=> support for iframes required Let's merge this issue into https://github.com/kaliiiiiiiiii/undetected-playwright-python/issues/5