a one-liner with huge effect ;) for now, i could not encounter any negative side-effects about this in my scrapers.
for use in the parse stage when dealing with the include_paths param, it can be very useful to be able to reference exact xpaths to specific links (a tags) on a page, i.e. a tag with a specific css class for pagination.
in the current implementation, we cannot directly specify only the "next" page link, as in the xpath we would need to set .//div[@class="pagination"] so that memorious finds all a children
with this improvement, the "next" link can be directly specified via xpath .//a[@class="next"]
a one-liner with huge effect ;) for now, i could not encounter any negative side-effects about this in my scrapers.
for use in the
parse
stage when dealing with theinclude_paths
param, it can be very useful to be able to reference exact xpaths to specific links (a tags) on a page, i.e. a tag with a specific css class for pagination.consider this markup:
in the current implementation, we cannot directly specify only the "next" page link, as in the xpath we would need to set
.//div[@class="pagination"]
so that memorious finds alla
childrenwith this improvement, the "next" link can be directly specified via xpath
.//a[@class="next"]