How to improve the speed of HTTPS requests and responses?

[jaimbox]

I understand that if we use a VM located in the us-east-1 region the response speed may be faster than having it in another region.

In order to avoid IP blocking it is necessary to use a Proxy, for example ExpressVPN.

The problem is that by using a Proxy we are reducing the speed of http requests. What solution can be implemented to improve the speed of both request and response?

@lpcs007

[zmkaihan]

@jaimbox Residential proxy is a good option and another option that I am working on is the IP rotator Amazon API gateway it gives you us-east-1 IPs per each request but I do not know how to make that to one IP for each request

[lpcs007]

I am not using Proxy. But I'm running the bot on my home PC.

[zmkaihan]

@jaimbox Did you find anything or any idea how to do it please let me know I have the same issue

[jaimbox]

@zmkaihan you can to use https://justhackerthings.com/post/rotate-your-ip-address/

[zmkaihan]

@jaimbox I already use this (https://github.com/Ge0rg3/requests-ip-rotator) same Idea that you share but it changes the IP for each request and Amazon always gives me a response status code 403. Is there any possibility that it change the IP base on user and keep the IP for the user and for new user generate a new IP?

[jaimbox]

@jaimbox I already use this (https://github.com/Ge0rg3/requests-ip-rotator) same Idea that you share but it changes the IP for each request and Amazon always gives me a response status code 403. Is there any possibility that it change the IP base on user and keep the IP for the user and for new user generate a new IP?

Code 403 is the Captcha, there is another thread where it is talked about. https://github.com/mdesilva/AmazonFlexUnlimited/issues/144

[zmkaihan]

To clarify this CODE 403 (new access token has to be generated) and Code 307 is the captcha and I already solved the captcha.

My question is when I rotate IP for each request it through me a 403 error code per request to all users and IP rotate change the IP successfully but always gives me a 403 error code.

[jaimbox]

@zmkaihan You're right, I confused it. But nevertheless I am using this https://github.com/Ge0rg3/requests-ip-rotator and it works perfectly for me. The IP changes for each Request but I don't receive any 403.

I know that the Captcha is not from this Issue but could you indicate your solution in the other Issue.

Thank you

[zmkaihan]

I'm using the same to rotate IP Can you please look at my code if something is wrong that why I receive 403 error code

self.gateway = ApiGateway("https://flex-capacity-na.amazon.com", regions=["us-east-1"],access_key_id="My_id", access_key_secret="My_secret_key") self.gateway.start() self.session_1 = requests.Session() self.session_1.mount("https://flex-capacity-na.amazon.com", self.gateway)

Any help will be really appreciated

[jaimbox]

I'm using the same to rotate IP Can you please look at my code if something is wrong that why I receive 403 error code

self.gateway = ApiGateway("https://flex-capacity-na.amazon.com", regions=["us-east-1"],access_key_id="My_id", access_key_secret="My_secret_key") self.gateway.start() self.session_1 = requests.Session() self.session_1.mount("https://flex-capacity-na.amazon.com", self.gateway)

Any help will be really appreciated

Did you run these commands to create the resources in AWS?

python3 fire.py --access_key "XXXXXXXXXXX" --secret_access_key "XXXXXXXX" --region "us-east-1" --command create --url https://api.amazon.com/

python3 fire.py --access_key "XXXXXXXXXXX" --secret_access_key "XXXXXXXX" --region "us-east-1" --command create --url https://flex-capacity-na.amazon.com/

[jaimbox]

@zmkaihan You must replace the url that is generated in your code.

[jaimbox]

@zmkaihan in your AWS account you should have something like this:

[zmkaihan]

Sorry for late reply I was able to run the command and Fireprox_amazon generated in my AWS API gateway and it generated a URL in my code so now how can I request through my code to change the IP per request to get offers?

[jaimbox]

@zmkaihan For example, I ran the comand for the url https://ifconfig.me/ and it generated this url https://hy818ob1n2.execute-api.us-east-1.amazonaws.com/fireprox/ Every time you refresh you can see a new IP.

[jaimbox]

@zmkaihan You must generate the ulr only for the domain and subdomain. For example: https://flex-capacity-na.amazon.com, when you get the response just replace it in the code, example: https://hy818ob1n2.execute-api.us-east-1.amazonaws.com/fireprox/GetOffersForProviderPost

[zmkaihan]

@jaimbox much appreciated it works now and I just received only 2 times 403 errors it seems to be working>

quick question do you know how can I print out the IP each time I request to get offers?

[jaimbox]

@jaimbox much appreciated it works now and I just received only 2 times 403 errors it seems to be working>

quick question do you know how can I print out the IP each time I request to get offers?

You can use this.

def get_ip():
    response = requests.get('https://api64.ipify.org?format=json').json()
    print("IP ADDRESS: ", response["ip"])
    return response["ip"]

[zmkaihan]

@jaimbox much appreciated

[ransel2016]

Para aclarar este CÓDIGO 403 (se debe generar un nuevo token de acceso) y el Código 307 es el captcha y ya resolví el captcha.

Mi pregunta es cuando roteo la IP para cada solicitud, a través de mí aparece un código de error 403 por solicitud para todos los usuarios y la rotación de IP cambia la IP con éxito pero siempre me da un código de error 403.

Hello Sir, were you able to solve the captcha?

[jessusmach]

@zmkaihan you can to use https://justhackerthings.com/post/rotate-your-ip-address/

This still works with the new changes that we did? @jaimbox

[russbakr]

do you do this just for the get_offer portion ? or also the accept offer request .. only cuz im also getting the 403