When a new window opens - skyvern fails to see it (I have a temporary solution)

[matan1905]

So I ran into this issue where a login window would open and I had to use it to login. I used Claude with some files I suspected and it was great at finding the right place to add changes:

in skyvern/webeye/actions/handler.py, this is an updated function:

async def handle_click_action(
        action: actions.ClickAction,
        page: Page,
        scraped_page: ScrapedPage,
        task: Task,
        step: Step,
) -> tuple[list[ActionResult], Page | None]:
    xpath = await validate_actions_in_dom(action, page, scraped_page)
    await asyncio.sleep(0.3)
    if action.download:
        return await handle_click_to_download_file_action(action, page, scraped_page), None

    # Click the button
    await chain_click(
        task,
        page,
        action,
        xpath,
        timeout=SettingsManager.get_settings().BROWSER_ACTION_TIMEOUT_MS,
    )

    # Check for newly opened windows
    new_window = await page.context.wait_for_event("page", timeout=5000)
    if new_window:
        return [ActionSuccess()], new_window

    return [ActionSuccess()], None

then there are some changes to the handle_action function that supported the tuple and in agent step there is this piece of code:

                results, new_page = await ActionHandler.handle_action(scraped_page, task, step, browser_state, action)
                if new_page:
                    # Update the browser_state with the new page
                    browser_state.page = new_page

I don't think the quality of the code is good enough for a PR, but this is a temporary solution for anyone who is looking for one

[suchintan]

Oh this is very interesting -- a popup modal to log into a website.

Thank you for sharing a workaround here! I'll add this to our backlog :)

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.