awslabs / aws-glue-libs

AWS Glue Libraries are additions and enhancements to Spark for ETL operations.
Other
635 stars 299 forks source link

Ability to use sso named profiles through aws-sdk-java version 2 #169

Open joshhamann opened 1 year ago

joshhamann commented 1 year ago

I currently have a work around set up to set env variables based on SSO's current (and expire-able) credentials. It would be nice to be able to utilize this repo with SSO named profiles. I believe it is tracked back to this issue (https://github.com/aws/aws-sdk-java-v2/issues/1970) which shows it looks like we need aws-sdk-java version >= 2.15.33 to be able to utilize.

I believe this is the pom.xml that is utilized (for glue-3.0) for instance:

I also think this is the pom that is referenced within that dependency:

Which shows: <aws.sdk.version>1.12.128</aws.sdk.version>

Is it possible to upgrade to aws-sdk-java version to something >= 2.15.33? Or even better, is there a way to adjust that value within the pom.xml which is on the s3 backed maven repository, so I can handle additional dependency issues as they come up (apache iceberg, for instance).

david-wb commented 1 year ago

I am facing the same problem. Did you resolve it?

joshhamann commented 1 year ago

No - I don't believe it can be resolved outside of this repo given the s3 backed pom.xml that needs editing. My solution was a workaround that sets AWS variables for the scope of the command I am firing based on SSO. So something like:

{script that sets AWS variables based on your named profile for the scope of this command} {your script} in the terminal worked for me. Hopefully that helps!

selfisch commented 1 year ago

Any update on this? We are facing the same issue.

barryoneill commented 1 year ago

A less awful way is to do this is to set this in ~/.aws/config:

[profile foo]
credential_process=aws configure export-credentials --profile=your-sso-profile

Then you should be able to use profile foo without the need for wrapping calls in scripts. It's not ideal but 🤷🏻‍♂️

rokity commented 8 months ago

The solutions of @barryoneill maybe it works. But I suggest to create a PR to use AWS_PROFILE_SSO variable as parameter to identify the SSO profile.