No longer working - Githubissues

farukalamai / yelp-scraper-scrapy-python

Yelp Restaurant data scraping using python, scrapy spider

MIT License

3 stars 2 forks source link

No longer working #1

Open frishtik opened 3 months ago

frishtik commented 3 months ago

Great idea for a project. However, I did follow all instructions & it does run perfectly fine, but just creates an empty CSV... Could it be an issue with website architecture changes? Your project was published not so long ago so it's odd.

farukalamai commented 3 months ago

yes, website architecture is changed. Try to learn about how to select class, id to work with web scraping.

frishtik commented 3 months ago

Got it! Thanks for the quick reply! One more thing: how do I set rotating proxies? Yelp seems to immediately block any scraping tries... I see that the settings.py is quite different from the ordinary scrapy settings, is that right? I tried to run scrapy-rotating-proxies with a proxy list I found online but it doesn't seem to work... Did you happen to play with this stuff yourself? I guess anyone who scrapes Yelp would face the same issue. Thanks anyway for this!