search

hardkoded / puppeteer-sharp

Headless Chrome .NET API
https://www.puppeteersharp.com
MIT License
3.24k stars 431 forks source link

Problem with BoundingBoxAsync and non-existing frame #2660

Open 400ochkov opened 2 weeks ago

400ochkov commented 2 weeks ago

I trying to get BoundingBoxAsync of element but faced error that frame not found. But this element located in another frame, so I don't understand how to fix this problem, may be anyone can help me

My code:

foreach (var frame in page.Frames)
       {
           Console.WriteLine(frame.Id);
           var elements = await frame.XPathAsync(".//select[@id='select']");
           Console.WriteLine("elements.Length in frame "+frame.Id+" = " + elements.Length);
           if (elements.Length > 0)
           {
               var boundingBox = await elements[0].BoundingBoxAsync();

               if (boundingBox != null)
               {
                   Console.WriteLine($"Frame URL: {frame.Url}");
                   Console.WriteLine($"X: {boundingBox.X}");
                   Console.WriteLine($"Y: {boundingBox.Y}");
                   Console.WriteLine($"Width: {boundingBox.Width}");
                   Console.WriteLine($"Height: {boundingBox.Height}");

                   break;
               }
           }
       }

Result of my code:

444398C884B7D228CE4744687E417C6E
elements.Length in frame 444398C884B7D228CE4744687E417C6E = 0
3DB4AC94A94EE8E6EA6DE6CAC34C7D82
elements.Length in frame 3DB4AC94A94EE8E6EA6DE6CAC34C7D82 = 1
Frame 8876375E8F13B2900CD53AFACF5D449E not found

So I dont understand where puppeteer take frame 8876375E8F13B2900CD53AFACF5D449E because my page have only frame 444398C884B7D228CE4744687E417C6E and 3DB4AC94A94EE8E6EA6DE6CAC34C7D82... And element located in 3DB4AC94A94EE8E6EA6DE6CAC34C7D82, but when I try to do BoundingBoxAsync on this element I faced this error

400ochkov commented 2 weeks ago

This issue happened when I reconnect to browser via ws. When I first time establish connection I run this code

   foreach (var frame in page.Frames)
   {
       Console.WriteLine(frame.Id);

   }

and get 3 frames E1FE87D9D79358CCEF06CFF273BF5753 0BE93872A9096CBD2B9A2F69B8075CA7 D3217171B894652C77C71892C2214762

But when I reconnect and run this code I get only 2 0BE93872A9096CBD2B9A2F69B8075CA7 D3217171B894652C77C71892C2214762

And if I want to interact with form as I wrote in first post I get error Frame E1FE87D9D79358CCEF06CFF273BF5753 not found. It means frame which was found at first connecction is missed after reconnection (but as I wrote before my element located not in this frame...)

Is this bug or I misunderstand something?

kblok commented 2 weeks ago

Can you share the exception's stack trace?

400ochkov commented 2 weeks ago

Can you share the exception's stack trace?

at PuppeteerSharp.Helpers.AsyncDictionaryHelper`2.<>c__DisplayClass6_0.<GetItemAsync>b__0() in /home/runner/work/puppeteer-sharp/puppeteer-sharp/lib/PuppeteerSharp/Helpers/AsyncDictionaryHelper.cs:line 35
   at PuppeteerSharp.Helpers.TaskHelper.WithTimeout[T](Task`1 task, Action timeoutAction, TimeSpan timeout) in /home/runner/work/puppeteer-sharp/puppeteer-sharp/lib/PuppeteerSharp/Helpers/TaskHelper.cs:line 146
   at PuppeteerSharp.Helpers.AsyncDictionaryHelper`2.GetItemAsync(TKey key) in /home/runner/work/puppeteer-sharp/puppeteer-sharp/lib/PuppeteerSharp/Helpers/AsyncDictionaryHelper.cs:line 33
   at PuppeteerSharp.Cdp.CdpElementHandle.ContentFrameAsync() in /home/runner/work/puppeteer-sharp/puppeteer-sharp/lib/PuppeteerSharp/Cdp/CdpElementHandle.cs:line 69
   at PuppeteerSharp.Frame.FrameElementAsync() in /home/runner/work/puppeteer-sharp/puppeteer-sharp/lib/PuppeteerSharp/Frame.cs:line 316
   at PuppeteerSharp.Frame.FrameElementAsync() in /home/runner/work/puppeteer-sharp/puppeteer-sharp/lib/PuppeteerSharp/Frame.cs:line 314
   at PuppeteerSharp.ElementHandle.GetTopLeftCornerOfFrameAsync() in /home/runner/work/puppeteer-sharp/puppeteer-sharp/lib/PuppeteerSharp/ElementHandle.cs:line 862
   at PuppeteerSharp.ElementHandle.<>c.<<BoundingBoxAsync>b__38_0>d.MoveNext() in /home/runner/work/puppeteer-sharp/puppeteer-sharp/lib/PuppeteerSharp/ElementHandle.cs:line 319
--- End of stack trace from previous location ---
   at PuppeteerSharp.ElementHandle.BindIsolatedHandleAsync[TResult,TElementHandle](Func`2 action) in /home/runner/work/puppeteer-sharp/puppeteer-sharp/lib/PuppeteerSharp/ElementHandle.cs:line 738
   at Program.Main(String[] args) in C:\Users\manwi\source\repos\puppeteer\Program.cs:line 123
kblok commented 2 weeks ago

@400 are you using the same Element handle after reconnection?

400ochkov commented 2 weeks ago

@400 are you using the same Element handle after reconnection?

not same, When reconnect I start programm again so method XPathAsync which return Elements collection started again and each restarting it returns element I think it's impossible to save Element handle or any type of object somewhere that continue to work with it after restarting programm

kblok commented 2 weeks ago

@400ochkov this test proves that we are able to recover all the iframes from a page.

If you have some code that I can run on my end. I would need some code I can run on my end.

400ochkov commented 2 weeks ago

proves that we are able to recover all the iframes from a page

as I understand this code make reconnect when connection lost. But in my case connection didn't lost, With connection everything is okay, but my programm finish and in order to work with puppeteer again I need to connect to this ws again when start my programm again

code like this

string ws = "ws://127.0.0.1:56570/devtools/browser/938e2b44-c3c0-4858-af79-93de84323733";

var webSocketDebuggerUrl = ws;

ViewPortOptions DefaultViewport = new ViewPortOptions();
DefaultViewport = (ViewPortOptions)null;

var browser = await Puppeteer.ConnectAsync(new ConnectOptions
{
    BrowserWSEndpoint = webSocketDebuggerUrl,
    DefaultViewport = DefaultViewport
});

var pages = await browser.PagesAsync();
var page = pages[0];

foreach (var frame in page.Frames)
           {
               console.writeline(frame.id);
               var elements = await frame.xpathasync(".//select[@id='select']");
               console.writeline("elements.length in frame " + frame.id + " = " + elements.length);
               if (elements.length > 0)
               {
                   var boundingbox = await elements[0].boundingboxasync();

                   if (boundingbox != null)
                   {
                       console.writeline($"frame url: {frame.url}");
                       console.writeline($"x: {boundingbox.x}");
                       console.writeline($"y: {boundingbox.y}");
                       console.writeline($"width: {boundingbox.width}");
                       console.writeline($"height: {boundingbox.height}");

                       break;
                   }
               }
           }

First run of programm I create antidetect browser, run it and get it websocket. Next time I take ws link and connect direct to it without creating

kblok commented 2 weeks ago

@400ochkov what's the page you're testing? I don't see that there. I need some program I can run on my end.

400ochkov commented 2 weeks ago

@400ochkov what's the page you're testing? I don't see that there. I need some program I can run on my end.

unfortenutaly on my site this frame with this form appears rarely, so I tried to find another site with same frames. Can't find same, but find one which had as I think related problem with frames. When I create browser and connect to it first time I got 7 frames. But when I reconnect I have only 3 (without exception)

First time I run this code`

//my methods to create antidetect browser profile
string profile_id = await dolp.new_profile("126");
string ws = dolp.start_profile(profile_id);

Console.WriteLine(ws);

//string ws = "ws://127.0.0.1:56570/devtools/browser/938e2b44-c3c0-4858-af79-93de84323733";

var webSocketDebuggerUrl = ws;

ViewPortOptions DefaultViewport = new ViewPortOptions();
DefaultViewport = (ViewPortOptions)null;

var browser = await Puppeteer.ConnectAsync(new ConnectOptions
{
    BrowserWSEndpoint = webSocketDebuggerUrl,
    DefaultViewport = DefaultViewport
});

var pages = await browser.PagesAsync();
var page = pages[0];
await page.GoToAsync("https://codesandbox.io/p/sandbox/funcaptcha-demo-s8srw1?file=%2Findex.html");
await Task.Delay(11000);

foreach (var frame in page.Frames)
           {
               console.writeline(frame.id);
               var elements = await frame.xpathasync(".//div[@id='app']");
               console.writeline("elements.length in frame " + frame.id + " = " + elements.length);
               if (elements.length > 0)
               {
                   var boundingbox = await elements[0].boundingboxasync();

                   if (boundingbox != null)
                   {
                       console.writeline($"frame url: {frame.url}");
                       console.writeline($"x: {boundingbox.x}");
                       console.writeline($"y: {boundingbox.y}");
                       console.writeline($"width: {boundingbox.width}");
                       console.writeline($"height: {boundingbox.height}");

                       //break;
                   }
               }
           }

And got 7 frames in result

ws://127.0.0.1:56873/devtools/browser/cd099399-a3e8-4953-a5c0-95c4ff96601b
232D426F29B1D2E3C508F9E5617F33C1
elements.Length in frame 232D426F29B1D2E3C508F9E5617F33C1 = 0
2D27FFC819033AC5533611640380206D
elements.Length in frame 2D27FFC819033AC5533611640380206D = 0
3D22A06477BAF80AEABB86702C20C946
elements.Length in frame 3D22A06477BAF80AEABB86702C20C946 = 0
AE34E2AE80B3EC78FCDFEC7F486EDE89
elements.Length in frame AE34E2AE80B3EC78FCDFEC7F486EDE89 = 0
4424736C6615D8C76A043DE93B2C92F6
elements.Length in frame 4424736C6615D8C76A043DE93B2C92F6 = 1
Frame URL: https://client-api.arkoselabs.com/v2/2.6.1/enforcement.bf6c920f19fb49ce16c82aaef4b83f45.html#11111111-1111-1111-1111-111111111111&3d746aec-faa0-4961-b361-e1b4c16d1d49
X: 784,109375
Y: 450,375
Width: 400
Height: 450
23879651890BD553D32200AD66230D93
elements.Length in frame 23879651890BD553D32200AD66230D93 = 0
8776A68B6407B36F3EC0646DD8AE9FB0
elements.Length in frame 8776A68B6407B36F3EC0646DD8AE9FB0 = 0

After that I tried to connect to ws and get frames again (didn't do anything with browser)

//my methods to create antidetect browser profile
//string profile_id = await dolp.new_profile("126", proxyManager);
//string ws = dolp.start_profile(profile_id);

//Console.WriteLine(ws);

string ws = "ws://127.0.0.1:58566/devtools/browser/93bc36f3-9499-43e8-8195-558bfa3f12fd";

var webSocketDebuggerUrl = ws;

ViewPortOptions DefaultViewport = new ViewPortOptions();
DefaultViewport = (ViewPortOptions)null;

var browser = await Puppeteer.ConnectAsync(new ConnectOptions
{
    BrowserWSEndpoint = webSocketDebuggerUrl,
    DefaultViewport = DefaultViewport
});

var pages = await browser.PagesAsync();
var page = pages[0];
//await page.GoToAsync("https://codesandbox.io/p/sandbox/funcaptcha-demo-s8srw1?file=%2Findex.html");
// await Task.Delay(11000);

foreach (var frame in page.Frames)
           {
               console.writeline(frame.id);
               var elements = await frame.xpathasync(".//div[@id='app']");
               console.writeline("elements.length in frame " + frame.id + " = " + elements.length);
               if (elements.length > 0)
               {
                   var boundingbox = await elements[0].boundingboxasync();

                   if (boundingbox != null)
                   {
                       console.writeline($"frame url: {frame.url}");
                       console.writeline($"x: {boundingbox.x}");
                       console.writeline($"y: {boundingbox.y}");
                       console.writeline($"width: {boundingbox.width}");
                       console.writeline($"height: {boundingbox.height}");

                       //break;
                   }
               }
           }

Result: 

2D27FFC819033AC5533611640380206D
elements.Length in frame 2D27FFC819033AC5533611640380206D = 0
AE34E2AE80B3EC78FCDFEC7F486EDE89
elements.Length in frame AE34E2AE80B3EC78FCDFEC7F486EDE89 = 0
23879651890BD553D32200AD66230D93
elements.Length in frame 23879651890BD553D32200AD66230D93 = 0
8776A68B6407B36F3EC0646DD8AE9FB0
elements.Length in frame 8776A68B6407B36F3EC0646DD8AE9FB0 = 0
kblok commented 2 weeks ago

I was able to reproduce it!

400ochkov commented 2 weeks ago

I was able to reproduce it!

is it possible to fix this issue?

kblok commented 2 weeks ago

Reported upstream https://github.com/puppeteer/puppeteer/issues/12613

400ochkov commented 1 week ago

Reported upstream puppeteer/puppeteer#12613

in this topic OrKoN said we need maintain the tree relationship between CDP-targets. Does it possible to maintan it in PuppeteerSharp?

kblok commented 1 week ago

Reported upstream puppeteer/puppeteer#12613

in this topic OrKoN said we need maintain the tree relationship between CDP-targets. Does it possible to maintan it in PuppeteerSharp?

Yes. We need to see how they will fix it.