Overmiind / Puppeteer-sharp-extra

Plugin framework for PuppeteerSharp
MIT License
183 stars 51 forks source link

Protocol error (Page.addScriptToEvaluateOnNewDocument): Session closed #21

Closed gfaessler closed 3 years ago

gfaessler commented 3 years ago

I updated to latest version and with this version I regularly get following error :


Unhandled exception. PuppeteerSharp.PuppeteerException: Protocol error (Page.addScriptToEvaluateOnNewDocument): Session closed. Most likely the Page has been closed.Close reason: Target.detachedFromTarget
   at PuppeteerSharp.CDPSession.SendAsync(String method, Object args, Boolean waitForCallback) in C:\projects\puppeteer-sharp\lib\PuppeteerSharp\CDPSession.cs:line 157
   at PuppeteerExtraSharp.Plugins.ExtraStealth.Utils.EvaluateOnNewPageWithUtilsScript(Page page, String script, Object[] args)
   at PuppeteerExtraSharp.PuppeteerExtra.<>c__DisplayClass9_1.<<Register>b__2>d.MoveNext()
--- End of stack trace from previous location ---
   at System.Threading.Tasks.Task.<>c.<ThrowAsync>b__140_1(Object state)
   at System.Threading.QueueUserWorkItemCallbackDefaultContext.Execute()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()

It seems to be an issue between PuppeteerExtra and PuppeteerSharp. I will try to make a simple reproducable case
Overmiind commented 3 years ago

Hi, I'm waiting for your reproducible case

gfaessler commented 3 years ago

Hello, following code triggers the issue.

If I remove the using statement on page or run each request on 1 thread one by one it works. Following code works using only Puppeteersharp or previous Pupeteersharp-Extra version. It looks like that on last version and in a multi-threaded environment some references remain on page object and it triggers an error when page object is disposed.


public static async Task Main(string[] args)
{
    int parallelismDegree = 20;
    List<string> sources = new List<string>();

    for (int i = 0; i < 200; i++)
    {
        sources.Add("https://www.refugederborence.ch/?=" + i.ToString() );
    }

    // Initialization plugin builder
    var extra = new PuppeteerExtra();

    // Use stealth plugin
    extra.Use(new StealthPlugin());

    // Launch the puppeteer browser with plugins
    var version = new BrowserFetcher().DownloadAsync().Result;

    // Launch the puppeteer browser with plugins
    using (var browser = await extra.LaunchAsync(new LaunchOptions { Headless = true, ExecutablePath = version.ExecutablePath, IgnoreHTTPSErrors=true }))
    {
        int sourceIndex = 0;
        while (sourceIndex < sources.Count)
        {
            Console.WriteLine(string.Format("Running {0} tasks from startIndex : {1}", parallelismDegree, sourceIndex));
            var tasks = sources.Skip(sourceIndex).Take(parallelismDegree).Select(async source =>
            {
                // Create a new page
                using (var page = await browser.NewPageAsync())
                {
                    await page.GoToAsync(source);
                    Console.WriteLine(string.Format("Running task : {0}", source));
                }

            });

            await Task.WhenAll(tasks);
            sourceIndex += parallelismDegree;
        }
    }
    Console.WriteLine("Press a key to continue");
    Console.ReadLine();
}
Ennorath commented 3 years ago

Good day! Is there any solution of it? Because, I also cannot start more then one browser instance at the same time, due this problem. Same error.

Overmiind commented 3 years ago

Hi, sorry to keep you waiting, I will fix it in a few days

Overmiind commented 3 years ago

Should be fixed by #26. Please update your NuGet package version to 1.3.1