search

john-hu / untitled

0 stars 0 forks source link

handle redirected URL #105

Open john-hu opened 2 years ago

john-hu commented 2 years ago 
2021-12-29 21:52:51 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (301) to <GET https://www.101cookbooks.com/kale-and-pecorino-salad/> from <GET https://www.101cookbooks.com/archives/lacinato-kale-and-pecorino-salad-recipe.html>
2021-12-29 21:53:03 [scrapy.extensions.logstats] INFO: Crawled 14 pages (at 1 pages/min), scraped 0 items (at 0 items/min)
2021-12-29 21:53:35 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (301) to <GET https://www.101cookbooks.com/pluot-summer-salad/> from <GET https://www.101cookbooks.com/archives/pluot-summer-salad-recipe.html>