aleksasiriski
commented
11 months ago Not worth it, because it introduces more network requests thus making the response time greater. It's better to have more instances deployed on different IPs (anycast is fine) and to work on better caching and making it less obvious that this is a scraper and not a human.
Describe your feature request Create a network of rotating proxies for avoiding rate limiting and throttling.
Additional context Heavily test if this slows down the search (since it has added network latency). It may not be worth it, but instead deploy more instances in the same region.