search

aschuma / aws-ec2-socks-proxy

Python scripts for provisioning an AWS EC2 instance and managing its lifecycle. Furthermore, the scripts return a shell command that assists the user in initiating a local Ssh SOCKSv5 proxy. This proxy routes all requests through the managed AWS EC2 instance, thereby enabling users to circumvent Geo-IP restrictions.
Apache License 2.0
5 stars 0 forks source link

Proxy Inquiry #24

Closed ak3zaidan closed 2 months ago

ak3zaidan commented 2 months ago

Hello I have a very simple question. I currently spend a ton of money renting out ISP or residential proxies that come in the form

IP Address;Port Number; Login ID;Password

You seem to be quite knowledgable in this field. I'm not sure if this repository accomplishes such thing but I'd appreciate it if you can point in the right direction. I'd like to create my own proxies for e-commerce use. I'm looking to create a large amount of them too. I'd appreciate any tips. Thanks.

aschuma commented 2 months ago

Just a short answer (I think I didn't fully understand your question either)

If your knowledge about networks is limited, sticking with a commercial VPN is the best option because it’s easy to use and doesn’t require much technical setup.

However, if you’re comfortable with some basic server management, consider renting a tiny VPS server for each relevant region (costing around 2€ per month for each region) and installing an OpenVPN servers there. This gives you more control. Beside browser traffic, other apps like email clients might use the infrastructure too. You have to install an open von client on your laptop/pc too. The software is open source, and you don't have to install any untrustworthy third-party software.

If you only need ad hoc access to bypass geo-IP restrictions while using a browser, then using an SSH SOCKS tunnel or the software maintained in this repository can be a simple and affordable solution.

ak3zaidan commented 2 months ago

Thanks for the reply. I understand my question was not very straightforward, let me explain. I have an automation bot that purchases products for me at very high rates. To prevent Shopify API request limiting I use a different proxy for each request, this basically makes the website think that each request is from a unique user. The website I added below is where I buy metered and unmetered proxies from, and I can pick the region where the proxies originate from. My use case is not really by passing geo-IP restrictions as you said. I'm not exactly sure how to set up a service like the website below, but I am assuming it is similar to as you mentioned, renting out servers for each region. What are the steps for this? Are there any links you can send that show a tutorial of this.

I'm not really experienced in this but I want to learn. Could I take this route: AWS for small VPS provider, Install Squid or 3Proxy software, deploy proxies with Terraform.

I realistically want to be able to create thousands of unique proxies, whats the best approach with this? Would I need to do something along the lines of docker containerization.

https://liveproxies.io/rotating-residential-proxies-pricing

aschuma commented 2 months ago

Your project is quite interesting, and you have a decision to make, whether to build your own solution or buy existing services as before.

AWS can definitely help you create many unique proxies using services like EC2 and assign multiple ip's for each instance. However, AWS can be expensive depending on number of services you actually deploy. I suggest checking AWS's pricing page to see whether it fits.

So, If AWS pricing works for you, there are several ways to automate the setup of these instances: Terraform, Ansible, AWS CLI, or Boto3, depending on what you're comfortable with. Squid is a good option to establish routing.

I don’t want to give you specific advice on which technology or stack to use because I’d need to dive deeper into the APIs and your exact requirements. My spare time is limited. I recommend asking for advice on forums like Reddit or Stack Overflow to get input from a wider audience.

Additionally, ChatGPT can help you get some initial pointers on which direction to take or provide basic implementation examples. However, be warned, some AWS APIs can be tricky, and you might not always get the results you expect, when asking AI. This is especially true for services like S3, though that's beyond the scope of this discussion. So, IMHO checking the AWS documentation or tutorials is a must. For Boto3, you can start with the examples provided here: https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python

I'm really interested to see what solution you end up choosing. Good luck!