Searching with coordinates gets blocked - Error 403

johnbalvin / pyzill

Zillow scraper made in Python

MIT License

21 stars 4 forks source link

Searching with coordinates gets blocked - Error 403 #2

Closed Merci93 closed 3 months ago

Merci93 commented 3 months ago

Searching with coordinates seems to get blocked at some point. Probably an IP address blocking or something since this calls the backend API. Throws an Error 403 and stops working. Any way this can be remedied?

Merci93 commented 3 months ago

Figured it out.

Zillow aside having an anti-bot scraper, implemented rate limiting on their end. Too much requests per second gets flagged as bot and the IP get's temporarilly blocked. Adding a random generator for sleep time between each requests solved this.

johnbalvin commented 3 months ago

Hi, yes it seems the issue is IP, you'll need to use proxies or use delays like you said