Protocol error (Runtime.evaluate): Session closed. Frame ... not found

[gorzko]

Description

I'm using PuppeteerSharp in an RPA (Blue Prism) project for some specific things, mostly invoking JS. With most pages it works just fine, but there is an issue with SAP Concur. When webpage is navigated outside of Puppeteer (manually or by Blue Prism) I get one of these errors:

Execution context was destroyed, most likely because of a navigation.

PuppeteerException:` Protocol error (Runtime.evaluate): Session closed. Most likely the Page has been closed.Close reason: Connection failed to process Runtime.executionContextCreated. Frame 6EA76FD7E7417DC0F62D5B1BBFB4E2FA not found.    at PuppeteerSharp.Helpers.AsyncDictionaryHelper`2.<>c__DisplayClass4_0.<GetItemAsync>b__0() in C:\projects\puppeteer-sharp\lib\PuppeteerSharp\Helpers\AsyncDictionaryHelper.cs:line 34
   at PuppeteerSharp.Helpers.TaskHelper.<WithTimeout>d__6`1.MoveNext() in C:\projects\puppeteer-sharp\lib\PuppeteerSharp\Helpers\TaskHelper.cs:line 129
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at PuppeteerSharp.Helpers.AsyncDictionaryHelper`2.<GetItemAsync>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at PuppeteerSharp.FrameManager.<OnExecutionContextCreatedAsync>d__57.MoveNext() in C:\projects\puppeteer-sharp\lib\PuppeteerSharp\FrameManager.cs:line 316
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at PuppeteerSharp.FrameManager.<>c__DisplayClass49_0.<<Client_MessageReceived>b__0>d.MoveNext() in C:\projects\puppeteer-sharp\lib\PuppeteerSharp\FrameManager.cs:line 202

First exception can be easily solved by executing Connect / AttachPage again, but then I always land with the second one.

Complete minimal example reproducing the issue

Main code 1:

// Browser is started by running: msedge "https://www.concursolutions.com" -remote-debugging-port=21222
// Then it is navigated by RPA/human to https://www.concursolutions.com/companyadmin/company_admin.asp
Chromium_Puppeteer browser = new Chromium_Puppeteer();
string r;
browser.Connect("http://127.0.0.1:21222").GetAwaiter().GetResult();
browser.AttachPage("https://www.concursolutions.com/companyadmin/company_admin.asp").GetAwaiter().GetResult();
// There is a breakpoint at this point and link is clicked by external tool/human and webpage navigates to https://www.concursolutions.com/companyadmin/view_users.asp
r = browser.EvaluateExpression("alert('!')").GetAwaiter().GetResult();

Main code 2:

// Browser is started by running: msedge "https://www.concursolutions.com" -remote-debugging-port=21222
// Then it is navigated by RPA/human to https://www.concursolutions.com/companyadmin/company_admin.asp
Chromium_Puppeteer browser = new Chromium_Puppeteer();
string r;
// There is a breakpoint at this point and link is clicked by external tool/human and webpage navigates to https://www.concursolutions.com/companyadmin/view_users.asp
browser.Connect("http://127.0.0.1:21222").GetAwaiter().GetResult();
browser.AttachPage("https://www.concursolutions.com/companyadmin/view_users.asp").GetAwaiter().GetResult();
r = browser.EvaluateExpression("alert('!')").GetAwaiter().GetResult();

My helper class:

internal class Chromium_Puppeteer
{
    private Browser browser; private Page page;

    public async Task Connect(string browserUrl)
    {
        browser = await Puppeteer.ConnectAsync(new ConnectOptions
        {
            BrowserURL = browserUrl,
            DefaultViewport = null
        });
    }

    public async Task AttachPage(string url)
    {
        var pages = await browser.PagesAsync();
        page = pages.Where(p => p.Url.Contains(url)).First();
    }

    public async Task<string> EvaluateExpression(string script)
    {
        var result = await page.EvaluateExpressionAsync(script);
        return result != null ? result.ToString() : "";
    }
}

Expected behavior:

If webpage is navigated externally after Connect / AttachPage, exception can be expected. But if it is navigated before browser and page are initialized, it should invoke JS.

Versions

• PuppeteerSharp 7.1.0 (but tried on 8.0.0 with the same result)
• .NET framework 4.8

Additional Information

During debuging I found out that before external navigation page had three frames:

AEB35F2284C3A94898FB1C9B34183071
9FA36C8100CB09323F108B8EFE6B26C7
01AD3E67B3BE9DC45D3BA3F1E1F153F6

After navigation it also had three, with two of them with different IDs

D66722F113434FB62A4B6B2FAEA33DB0
9FA36C8100CB09323F108B8EFE6B26C7
C466CFB8CF6028EEA166D2DE778EE7BC

However in exception there is frame 6EA76FD7E7417DC0F62D5B1BBFB4E2FA mentioned... where did it come from?

[FahadAltaf]

I think latest release of chrome driver causing this issue

[kblok]

I'm working on a Target manager refactor that might help here.

[amaitland]

There are two ways Frames are added to the ConcurrentDictionary, only the frames created in OnFrameNavigatedAsync will resolve the Task created by GetItemAsync. The frames created in OnFrameAttached are added to the ConcurrentDictionary directly and never resolve any pending Tasks.

If someone wants to try out a potential fix it's available at https://github.com/amaitland/puppeteer-sharp/commit/6983ba0c0712a59598df72c56f5894847bbe80c6

[kblok]

@amaitland Do you think we can have a PR with that change?

[amaitland]

Sure, PR #2049 created

[amaitland]

@gorzko Can you try out version 9.0? Hopefully this should be resolved.

[gorzko]

@amaitland @kblok After update to the last version, my program deadlocks on browser = await Puppeteer.ConnectAsync(new ConnectOptions { BrowserURL = browserUrl, DefaultViewport = null });.

[amaitland]

@gorzko Probably related to #2051

If you can create a self contained example that reproduces the problem then I'd be creating a new issue.

[gorzko]

@amaitland I'm not sure if that's what's needed, but that's a code that reproduces the issue:

using PuppeteerSharp;
using System.Diagnostics;
using System.Linq;
using System.Threading.Tasks;

namespace PuppeteerStuck
{
    internal class Program
    {
        private static void Main(string[] args)
        {
            Process.Start("msedge", "\"https://www.google.com\" -remote-debugging-port=21222");
            while (true)
            {
                if (Process.GetProcessesByName("msedge").Where(p => p.MainWindowTitle.StartsWith("Google")).Any())
                {
                    break;
                }
                else
                {
                    System.Threading.Thread.Sleep(1000);
                }
            }
            ConnectBrowser().GetAwaiter().GetResult();
        }

        private static async Task ConnectBrowser()
        {
            var browser = await Puppeteer.ConnectAsync(new ConnectOptions
            {
                BrowserURL = "http://127.0.0.1:21222",
                DefaultViewport = null
            });
        }
    }
}

Application gets stuck on var browser = ... in PuppeteerSharp 9.0.x. On versions 7.1.0 and 8.0.0 it works fine.

[amaitland]

@gorzko Best to create a new issue as that seems unrelated to this issue.