Sleep statements needed in code to avoid errors

[sdondley]

I've got chromium installed on a Debian 10 box with 8 GB of ram. I will frequently, but not always, see errors unless I place sleep statements in the code like so:

#!/usr/bin/perl
use File::Temp 'tempdir';
use Time::HiRes 'usleep';
use Log::Log4perl qw(:easy);
use WWW::Mechanize::Chrome;

Log::Log4perl->easy_init($ERROR);
my $mech = WWW::Mechanize::Chrome->new(sync => 1, launch_exe => '/usr/bin/chromium', headless => 1, incognito => 1, data_directory => tempdir( CLEANUP => 1 ), profile => '/home/admin');
$mech->get('http://exmaple.com/wp-login.php');

usleep 500000;
$mech->submit_form( with_fields => { log => 'user', pwd => 'pass' } );

usleep 500000;
print $mech->content;

Here's an example of the errors I'll get without the sleep statements:

Bad luck: Node with nodeId 0 found. Info for this one cannot be retrieved at /usr/local/share/perl/5.28.1/WWW/Mechanize/Chrome.pm line 4026.
No node with given id found

-32000 at /usr/local/share/perl/5.28.1/Chrome/DevToolsProtocol/Target.pm line 502
Node 0 has gone away in the meantime, could not resolve at /usr/local/share/perl/5.28.1/WWW/Mechanize/Chrome/Node.pm line 206.
Can't call method "send_message" on an undefined value at /usr/local/share/perl/5.28.1/WWW/Mechanize/Chrome/Node.pm line 95.

When this error occurs, the HTML does not get printed to the screen at all as execpted.

A less frequent error I see is this one:

No search session with given id found

-32000 at /usr/local/share/perl/5.28.1/Chrome/DevToolsProtocol/Target.pm line 502

Other times, the script will run but will print this error before printing the content of the page:

No node with given id found

-32000 at /usr/local/share/perl/5.28.1/Chrome/DevToolsProtocol/Target.pm line 502
Node 157 has gone away in the meantime, could not resolve at /usr/local/share/perl/5.28.1/WWW/Mechanize/Chrome/Node.pm line 206.

Are there any settings I can use to avoid these errors? I'd prefer not to have to litter my code with sleep statements.

[Corion]

Oh - I encounter these problems with nodes getting nodeId 0 , but I didn't know you can avoid them by sleeping a bit. I'll investigate whether sleeping within ->xpath prevents this problem.

If you have a reliable way of producing the problem, I would very much like that, as currently the problem occurs for me in maybe 1 of 20 test runs or so.

[sdondley]

It happens for me nearly 100% of the time in the code sample above. If I remove the $mech->submit_form line in the sample code, the error occurs much less frequently.

If it'll help, I can send you a user name, password and url for the site (it's just a staging site). How can I get those credentials to you securely?

[nico-8]

@Corion any news of this issue ? I have the same problems as @sdondley

[Corion]

Unfortunately, the experiment with sleeping did not help with the issue. I can replicate the issue, but have not found ways to fix the issue.

[nico-8]

Is this specific to Perl ? I mean there is implementation of Chrome automation in other language. Maybe a track to follow ? For me, I test with $mech->selector and $mech->xpath, I always have the issue.

[Corion]

I think I found the same error elsewhere [1], and the solution seems to be to call DOM.getDocument only once per page. I'm in the process of going through the codebase and reworking it, but I'm not sure if it really works. But thanks for the idea of looking at how others cope with this problem!

1

[nico-8]

Excellent news ! Hope it will work. If you need more inspiration, follow this 1. And of course, submit your Awesome work to this list 😎 I really want to use Perl to automate Chrome.

[nico-8]

@Corion have a look at Selenium::Chrome, it is working well for me. So maybe, it could be interesting to have a look how they implement it.