crwlrsoft / crawler

Library for Rapid (Web) Crawler and Scraper Development
https://www.crwlr.software/packages/crawler
MIT License
312 stars 11 forks source link

Fix tracking request end for redirected requests #118

Closed otsch closed 11 months ago

otsch commented 11 months ago

When a URL redirects, the trackRequestEndFor() method of the HttpLoader's Throttler instance is called only once at the end and with the original request URL.