Timeout for loading dynamic content

[lusy777]

I need to load dynamic content for listening events. After scrolling on a page I need some delay (6-7 sec) for loading dynamic content, but any delays don't work and the browser closes before loading starts. sleep() is just stop any actions on the page and using js - just skips it

$page->navigate($url)->waitForNavigation(Page::NETWORK_IDLE, 40);
 $page->evaluate("
   const element = document.querySelector('#paginate');
   element.scrollIntoView({ behavior: 'smooth', block: 'end' });

   const delay = 5000;
   setTimeout(() => {
        const elementCop = document.querySelector('#navFooter');
        elementCop.scrollIntoView({ behavior: 'smooth', block: 'end' });
   }, delay);
");
$page->waitForNavigation('DOMContentLoaded');

setTimeout doesn't work, sleep() - just stops actions, waitForNavigation - doing nothing how to make a delay correctly?

[enricodias]

sleep() will make the entire php process hangs for a set amount of seconds but the browser should remain active. waitForNavigation() works by sleeping a few milliseconds at the time and checking for specific load events from the browser. Those are page load events and not related to any specific content loaded dynamically in the page.

Even if your javascript code works as intended, the php will not wait for it to finish, it will not even know when that timeout function gets executed. The DOMContentLoaded event will not be triggered by this, the page is already loaded. You can try to sleep after sending the javascript code to the browser or use the waitUntilContainsElement() method. Note that scrolling doesn't happen instantly and if your sleep is exactly the same time as the javascript timeout, the php process will resume before the scroll action finishes.

[lusy777]

Thx for response! That's right, I checked all cases. I tried sleep()after sending js - loading spinner and no new content for any sleep interval. I tried waitUntilContainsElement() method, load dynamic content, but don't stop for selector with attribute, like div[data-index="5"] (definitely exists), it works only with class/id, but new content don't have new class/id, so that's my problem. Maybe I don't understand about attributes or is there a way to delay php execution?

[enricodias]

You can pass a Selector object directly to the waitUntilContainsElement method, try using a XPathSelector.

sleep delays the php execution. The process stops completely. Just the php proccess, not the chrome process.

[lusy777]

Very strange, for me sleep stops loading all js scripts on the page (after click or scroll - nothing, loading spinner and same status for all scripts - "pending" or "cancelled"), can't reload page or load another link - nothing is loading.

$page->evaluate($options['js']);
sleep(150);

Yes, I tried $page->waitUntilContainsElement('div[data-index="6"]'); or like XPathSelector - doesn't work for loaded DOM or dynamic elements by attributes, checked another website - works with loaded element by class/id.

[enricodias]

Did you really meant to sleep for 150 seconds? The script will probably lose the connection to the browser itself due to timeouts.

The waitUntilContainsElement should also work for dynamic elements. It uses the same evaluate method under the hood. The main usage for this method is exactly for that kind of problem. Your code must probably has some other thing causing those issues.

[lusy777]

The waitUntilContainsElement works for dynamic elements and would be enough solution if this function would accept selector attributes, because dynamic elements don't have new class/id. Also tried XPathSelector - attributes doesn't work for this function. 150 seconds - just for debugging, I need about 5-10 seconds while loading dynamic elements. So,

• sleep - just freezes the appearance of dynamic elements
• waitUntilContainsElement - doesn't work with attributes If would be the third case would be great