laughingman7743
commented
3 years ago Starting in version 2.1.0, JDBC driver options are now specified as arguments to the connect method. https://github.com/laughingman7743/PyAthenaJDBC/releases/tag/v2.1.0
connect(Profile= aws.profile_name,
S3OutputLocation = 's3://BUCKET/',
AwsRegion = aws.region_name
)See the documentation for JDBC driver options. https://s3.amazonaws.com/athena-downloads/drivers/JDBC/SimbaAthenaJDBC_2.0.13/docs/Simba+Athena+JDBC+Driver+Install+and+Configuration+Guide.pdf
It is recommended to use the Boto3 version rather than the JDBC version. https://github.com/laughingman7743/PyAthena
I'm having difficulty using the profile_name parameter unless the profile's name is "default" in the credentials file. This issue is similar to Issue 51, although it looks like the source code for connection.py has changed since the commit to resolve issue 51 - I wasn't able to find profile_name, token, etc. as arguments in the present source code I looked at. Regardless, those arguments are included in the Python code below.
The credentials file I'm using for AWS changes constantly due to security reasons, and incorporates multiple roles (with none of them named "default"). I'm hoping there's a way to use PyAthenaJBDC to account for changing keys and tokens for multiple profiles when connecting to Athena.
Thank you for your time!
Reproduce Issue:
I have a credentials file that looks like this (~/.aws/credentials):
[user-123] aws_access_key_id = aaaaaaaa aws_secret_access_key = bbbbbbbb aws_session_token = cccccccc
[user-456] aws_access_key_id = dddddddd aws_secret_access_key = eeeeeeee aws_session_token = ffffffff
The code below will not run with the credentials file above:
from pyathenajdbc import connect from boto3 import Session
aws = Session(profile_name = "user-123") credentials = aws.get_credentials().get_frozen_credentials()
connect(access_key = credentials.access_key, secret_key = credentials.secret_key, token = credentials.token, profile_name = aws.profile_name, s3_staging_dir = 's3://BUCKET/', AwsRegion = aws.region_name )
However, the code runs if I change the name of one credential to "default":
[default] aws_access_key_id = aaaaaaaa aws_secret_access_key = bbbbbbbb aws_session_token = cccccccc
[user-456] aws_access_key_id = dddddddd aws_secret_access_key = eeeeeeee aws_session_token = ffffffff
Then change the Python code to have profile_name as "default":
from pyathenajdbc import connect from boto3 import Session
aws = Session(profile_name = "default") credentials = aws.get_credentials().get_frozen_credentials()
connect(access_key = credentials.access_key, secret_key = credentials.secret_key, token = credentials.token, profile_name = aws.profile_name, s3_staging_dir = 's3://BUCKET/', AwsRegion = aws.region_name )