search

igorsimb / mp-monitor

Django app for scraping Wildberries
1 stars 0 forks source link

Implement anti-blocking measures #165

Open igorsimb opened 4 months ago

igorsimb commented 4 months ago

Great article: https://www.scrapingbee.com/blog/web-scraping-without-getting-blocked/ Best free proxies: https://www.scrapingbee.com/blog/best-free-proxy-list-web-scraping/#introduction

Rotate User-Agent by using tools:

igorsimb commented 4 months ago

Related to https://github.com/igorsimb/mp-monitor/issues/164