search

kafagy / fifa-FUT-Data

Web-scraping script that writes the data of all players from FutHead and FutBin to a CSV file or a DB
MIT License
76 stars 17 forks source link

problems with error 403 #8

Open Dextron11 opened 5 years ago

Dextron11 commented 5 years ago

Adding a proxy, generating a user agent and random delays do not help, after 1 day of work, futbin blocks

kafagy commented 4 years ago

I think you're going to need something like scrapy and you might need to add some proxies to be able to trick their monitoring tools , so that you can be able to keep parsing. The problem with my script is that it shows a specific behavior, which is going through their player list in order and page by page.