Recurring 403 error

[cia05rf]

Just discovered this package and it has huge promise, however I'm not sure if something on AWS has changed since the last release.

Using the below code i always get a 403 response.

import requests
from requests_ip_rotator import ApiGateway

gateway = ApiGateway(
    "https://www.google.com",
    regions=['eu-west-1', 'eu-west-2', 'eu-west-3'],
    access_key_id="MY_KEY",
    access_key_secret="MY_SECRET"
    )
gateway.start()

session = requests.Session()
session.mount("https://www.google.com", gateway)
response = session.get("https://www.google.com/search?q=random+search")
print(response.status_code)

[Ge0rg3]

Hey @cia05rf , thanks for using the lib. I think this is probably an AWS permission issue, as the same works for me:

In [11]: import requests
    ...: from requests_ip_rotator import ApiGateway
    ...:
    ...: gateway = ApiGateway(
    ...:     "https://www.google.com",
    ...:     regions=['eu-west-1', 'eu-west-2', 'eu-west-3']
    ...: )
    ...: gateway.start()
    ...:
    ...: session = requests.Session()
    ...: session.mount("https://www.google.com", gateway)
    ...: response = session.get("https://www.google.com/search?q=random+search", headers={"User-Agent": "Mozilla/5.0 (Wi
    ...: ndows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.134 Safari/537.36"})
    ...: print(response.status_code)
Starting API gateways in 3 regions.
Using 3 endpoints with name 'https://www.google.com - IP Rotate API' (0 new).
200

It could also just be Google blocking your request, as sometimes they'll respond 403 if receiving heavy traffic from cloud providers at any given time.

[Ge0rg3]

Sorry I should add that in my above example I added a user agent header, that may solve your problem too!

[cia05rf]

Thanks for the quick response. I'll give it a go. I'm new to AWS (I've always used Azure up until now) so an AWS permission issue would be about right.

It would be helpful on the readme to have a list of the AWS permissions you need to help the uninitiated like me.

Thanks

[Ge0rg3]

Usually they're fine by default, you'd only get an AWS permission error if you've changed anything manually 😄 did you try with the user agent?

[cia05rf]

I've changed a few things. In case ths happens to anyone later down the line this is what i did:

1. Added user agent
2. Then tinkered in AWS:
   • Created a group and user, assigned the user to the group
   • Assign a pollicy with the below config to that group:

     {
     "Version": "2012-10-17",
     "Statement": [
     {
         "Sid": "VisualEditor0",
         "Effect": "Allow",
         "Action": [
             "apigateway:POST",
             "apigateway:PUT",
             "apigateway:GET"
         ],
         "Resource": [
             "arn:aws:apigateway:*::/restapis",
             "arn:aws:apigateway:*::/restapis/*/resourses",
             "arn:aws:apigateway:*::/restapis/*/resourses/*",
             "arn:aws:apigateway:*::/restapis/*/resourses/*/methods/ANY"
         ]
     }
     ]
     }

[Ge0rg3]

Cool glad to hear its working