saketmugunda123
commented
10 months ago Rotating Proxy: smartproxy $7.50 for 3 IP addresses Based on what I've researched, there are two viable options for our rotating proxy service. The first one is the most popular smartproxy. It has been awarded the best value rotating proxy and has the numbers to back it up. The average response time for the proxy service is less than 0.6 seconds, an industry best. Further, the success rate of smartproxy is 99.47%, a mark that is also the highest in the market. On top of all that, there is an affordable plan that provides us 3 IP addresses for $7.50 a month. With reliability, smartproxy is viewed as one of the most reliable in the market and doesn't violate any international laws. Unfortunately, cheaper services like PacketStream doesn't allow for custom rotation, and duplicate IP address usually occur. With all of this, smartproxy's 3 IP addresses for $7.50 a month while providing efficient and reliable service makes it the best service for our needs @johndpjr
johndpjr
Context
A rotating proxy helps us make more web requests by changing our IP after a set time or after a certain number of requests. If we want our web scraper(s) to scale, we need to make requests appear as if multiple users are accessing company job boards, otherwise our IP will be blacklisted and we will not be able to access that site through that IP anymore.
TODO
Notes
This proxy service won't yet be used in our project since our web scraping volume is not large, but once we do have a better scraping system in place with multiple web scrapers, this will become necessary. It's good to be prepared with potential options already.