encode / databases

Async database support for Python. 🗄
https://www.encode.io/databases/
BSD 3-Clause "New" or "Revised" License
3.84k stars 261 forks source link

How do I make connection to RDS Proxy endpoint? #490

Open girisagar46 opened 2 years ago

girisagar46 commented 2 years ago

Has anyone tried to use RDS Proxy with IAM Authentication and Databases?

While using RDS Proxy with IAM authentication, we need to connect to the Database through the temporary IAM token which we can get from the RDS API generate_db_auth_token(DBHostname, Port, DBUsername, Region=None)

This will generate A presigned url which can be used as an auth token. which looks like:

mydb.123456789012.us-east-1.rds.amazonaws.com:3306/?Action=connect&DBUser=db_user&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIEXAMPLE%2Fus-east-1%2Frds-db%2Faws4_request&X-Amz-Date=20210123T011543Z&X-Amz-Expires=900&X-Amz-SignedHeaders=host&X-Amz-Signature=88987EXAMPLE1EXAMPLE2EXAMPLE3EXAMPLE4EXAMPLE5EXAMPLE6

In addition to this, we also need to pass ssl params and ssl-ca file path.

I am lost here because with PyMySQL it's relatively easy like this:

session = boto3.Session()
rds_client = session.client("rds")
password = rds_client.generate_db_auth_token(DBHostname=host, Port=port, DBUsername=user)
path_to_pem = f"{os.path.dirname(__file__)}/rds-ca-2019-root.pem"
pymysql.connect(
        host=host,
        port=port,
        user=user,
        password=password,
        database=database_name,
        ssl_ca=path_to_pem,
        charset="utf8mb4",
        cursorclass=pymysql.cursors.DictCursor,
    )

How can I translate this to the equivalent async Databases code?

girisagar46 commented 2 years ago

So far I've come up with the following but it's not working.

from urllib.parse import quote
from sqlalchemy.engine import URL

path_to_pem = f"{os.path.dirname(__file__)}/AmazonRootCA1.pem"
db_url = URL.create(
    drivername="mysql",
    username=mysql_user,
    password=quote(create_connection_token()),
    host=mysql_host,
    port=mysql_port,
    database=mysql_db,
    query={"ssl": "true", "ssl_cipher": path_to_pem}
)

But it's not working.

The error I'm getting is:

[ERROR] OperationalError: (2003, "Can't connect to MySQL server on 'proxy-rds-read-only-endpoint.endpoint.proxy-xxxxxxxxxxx.us-east-1.rds.amazonaws.com'")
Traceback (most recent call last):
  File "/var/task/lambda_function.py", line 116, in lambda_handler
    return response(event)
  File "/var/task/lambda_function.py", line 151, in response
    clusters = asyncio.run(get_cluster(client_id, start_week, num_of_week))
  File "/var/lang/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/var/lang/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/var/task/lambda_function.py", line 261, in get_cluster
    async with Database(get_db_url()) as database:
  File "/var/task/databases/core.py", line 133, in __aenter__
    await self.connect()
  File "/var/task/databases/core.py", line 88, in connect
    await self._backend.connect()
  File "/var/task/databases/backends/mysql.py", line 61, in connect
    self._pool = await aiomysql.create_pool(
  File "/var/task/aiomysql/pool.py", line 29, in _create_pool
    await pool._fill_free_pool(False)
  File "/var/task/aiomysql/pool.py", line 167, in _fill_free_pool
    conn = await connect(echo=self._echo, loop=self._loop,
  File "/var/task/aiomysql/connection.py", line 75, in _connect
    await conn._connect()
  File "/var/task/aiomysql/connection.py", line 519, in _connect
    raise OperationalError(2003,