search

Corion / WWW-Mechanize-Chrome

automate the Chrome browser
https://metacpan.org/release/WWW-Mechanize-Chrome
Artistic License 2.0
30 stars 12 forks source link

VERY slow on xpath getting TDs of a TR #66

Open aimass opened 2 years ago

aimass commented 2 years ago

Hi there!

If there's a mailing list pls. let me know and I'll post the discussion there.

This is taking 2 to 3 seconds on average: my @cells = $mech->xpath('.//td', node => $rows[$row_index]);

what is making it so slow ?? Is there a way to speed this?

TIA!

-- Alex

aimass commented 2 years ago

UPDATE: the time seems to be proportional to the number of rows. The 2-3 second timing is for a table with 450 rows and 12 columns. On a table with 120 rows, it's less than a second. Speculating: it would seem that relative to node xpath has some bug and may be scanning the whole table each time.

Corion commented 2 years ago

See also this post. Most helpful would be a short, self-contained script to reproduce the issue.

aimass commented 2 years ago

Sure thing, I'll try to replicate the issue with some public site that has a table with a few hundred records, and follow up here and on PM.BTW, I actually wrote that post on PM ;-)

I updated that post and added a workaround at the end combining W::M::Chrome xpath to find the table node and then HTML::Tree to parse the table. I think this actually a good approach and reduces the Chrome overhead anyways.

aimass commented 2 years ago

Here you go: https://perlmonks.org/?node_id=11148399