aws / aws-sdk-pandas

pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
https://aws-sdk-pandas.readthedocs.io
Apache License 2.0
3.87k stars 680 forks source link

endpoint_url #538

Closed patrick-muller closed 3 years ago

patrick-muller commented 3 years ago

Describe the solution you'd like As suggested by Abdel Jaidi , When we define a connection to Glue, Athena etc, we need to add a feature to specify the URL for Athena. For example:

athena_endpoint_url lakeformation_endpoint_url

maxispeicher commented 3 years ago

You should just be able to set the URL in the awswrangler.config. Example for athena:

import awswrangler as wr

wr.config.athena_endpoint_url = "my_athena_url"

It is also possible to set the URL through environment variables. You can check the global configuration tutorial for more information: https://github.com/awslabs/aws-data-wrangler/blob/main/tutorials/021%20-%20Global%20Configurations.ipynb

igorborgest commented 3 years ago

Exactly, it was address on issue #418 .

Thanks @patrick-muller and @maxispeicher !

igorborgest commented 3 years ago

p.s. @jaidisido is working on something specific to lakeformation that was not supported yet. Butbesides this one I think we already are well covered for most of the popular services related to our features.

jaidisido commented 3 years ago

Yes Athena is already covered and Lake Formation will be in my PR