microsoft / playwright

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
https://playwright.dev
Apache License 2.0
66.95k stars 3.67k forks source link

[BUG]await can't work normal #19238

Closed jiangyangcreate closed 1 year ago

jiangyangcreate commented 1 year ago

Context:

Code Snippet

from playwright.async_api import async_playwright
import asyncio
import logging
logging.basicConfig(filename='1.log')
class TestA:
    def __init__(self,loop) -> None:
        asyncio.set_event_loop(loop=loop)

    async def asy_browser(self):
        p = await async_playwright().start()
        self.browser = await p.chromium.launch(headless=False)

    async def run_page(self,tid):
        context = await self.browser.new_context()
        page = await context.new_page()
        while True:
            await asyncio.sleep(1)
            try:
                await page.goto(f"https://{tid}")
                await asyncio.sleep(1)
            except:
                await page.close()
                await context.close()
                await asyncio.sleep(1)
                context = await self.browser.new_context()
                page = await context.new_page()
                logging.warning(f'{tid}close')

def test():
    async def spider(task_obj):
        await task_obj.asy_browser()
        async_task =  []
        for i in range(30):
            async_task.append(asyncio.ensure_future(task_obj.run_page(i)))
        await asyncio.wait(async_task)
    get_async_loop = asyncio.new_event_loop()
    asyncio.set_event_loop(get_async_loop)
    task_obj = TestA(get_async_loop)
    sy_t = asyncio.run_coroutine_threadsafe(spider(task_obj), loop=get_async_loop)
    get_async_loop.run_forever()
    return sy_t.result()

test()

Describe the bug

I set one browser and different context and i try to use page.goto for different url . if yes :continue go other url (for show the bug : i use simple url to define tid ) else close the page and context ,and create new page and context

task run and use await switch task when i put more task , will be have some task cannot resume .

1.log

i record by logging.warning(f'{tid}close') now i have 30 task and 300 log ,but [7:close] just 4 times

you can use headless=False , and will find some page longtime no close() it Lose the meaning of the async/await

jiangyangcreate commented 1 year ago

At first, I suspected it was a CPU problem ,CPU is not fast enough , but my cpu was just use 80%

mxschmitt commented 1 year ago

This is not an end-to-end specific use-case and looks more like a general Python related issue. Closing for now since we can't act on that without a clear small repro. I'd recommend to ask on SlackOverflow or GitHub Discussions.