search

orangecoding / fredy

:heart: Fredy - [F]ind [R]eal [E]states [D]amn Eas[y] - Fredy will constantly search for new listings on sites like Immoscout or Immowelt and send new results to you, so that you can focus on more important things in life ;)
http://www.orange-coding.net
MIT License
209 stars 54 forks source link

Setting Interval and Working Hours job-wise or provider-wise #55

Closed computerjanes closed 2 years ago

computerjanes commented 2 years ago

It would be awesome to set the interval and working hours specific for every job or even every provider. This would allow to use the free 10000 API Credits of ScrapingAnt most efficiently, while crawling all other sites on a higher interval.

orangecoding commented 2 years ago

I know this might be helpful, but quite frankly speaking, I would find it very unfair, to implement something like this. You know the folks @scrapingAnt do hard work to make sure scraping of immoscout works. There's even a 5% discount (see readme) If you need more than the 10k calls, consider subscribing for as long as you need the scraper.

orangecoding commented 2 years ago

@computerjanes fyi, I talked to the creator of ScrapingAnt. They managed to overcome the protection of Immoscout using datacenter proxies, which means you should have plenty of api calls left. however keep in mind, supporting them is an option...