awslabs / amazon-emr-vscode-toolkit

A VS Code Extension to make it easier to manage and develop Spark jobs on EMR
https://marketplace.visualstudio.com/items?itemName=AmazonEMR.emr-tools
Apache License 2.0
29 stars 4 forks source link

Extension cannot access local aws account #17

Closed Jeppefs closed 1 year ago

Jeppefs commented 1 year ago

Hey,

The extension does not have access to the local aws account. The aws account is configured and other tools have access to it such as the aws tools extensions. I am not sure what to do from here and have followed the guide https://marketplace.visualstudio.com/items?itemName=AmazonEMR.emr-tools&ssr=false#qna

Error fetching EMR clusters!CredentialsProviderError: Could not load credentials from any providers

dacort commented 1 year ago

Unfortunately this extension uses whatever your default AWS profile is or can also use environment variables exported in your shell if you start VS Code from the terminal. I understand this is a little confusing, so definitely considering options.

To help more, can you let me know:

As an example, I'm on macos and in a terminal can use the following commands to launch VS Code with the correct profile.

# Set my session credentials
export AWS_ACCESS_KEY_ID=AKIAxxxyyy
export AWS_SECRET_ACCESS_KEY=GEcttttjjjj
export AWS_SESSION_TOKEN=IQdfdfdkkk

# Start VS Code in a new window
code -n 

If I have a default profile defined in ~/.aws/config or ~/.aws/credentials it can also use those credentials.

dacort commented 1 year ago

@Jeppefs I just pushed a new version of the extension (0.5.0) that allows you to select an AWS profile with the "EMR: Select AWS Profile" command.

Once that hits the marketplace, feel free to update and give that a shot.

Going to close this for now, but feel free to reopen with more detail if it still doesn't work!

Jeppefs commented 1 year ago

Hi and thanks for the quick update @dacort

Sorry for a late response.

Here is answers to your questions: What OS you're on?

The new select profile feature is very nice, and it can detect all profiles defined in .aws/config. I do get an error when chossing a profile:

Error fetching EMR Serverless applications!AccessDeniedException: User: arn:aws:sts::000306902459:assumed-role/dev-user/aws-sdk-js-1674649492375 is not authorized to perform: emr-serverless:ListApplications on resource: arn:aws:emr-serverless:us-east-1:000306902459:/*

The profile reader seems to read the profile incorrectly as it uses the incorrect region. us-east-1 should be eu-west-1 as shown in the config. The profile is authorized to perform ListApplications as I am able to do so through AWS cli.

Could the error be based on the fact that we use SSO login. There is no acces_keys defined, and if your profile reader expects these to be defined in ~/.aws/config, ~/.aws/credentials or in the environment variables it fails to connect to aws with the correct profile.

Moreover when using EMR: Select AWS Region I get the following error:

Command 'EMR: Select AWS Region' resulted in an error (AWS was not able to validate the provided access credentials)
dacort commented 1 year ago

Interesting thanks for the details @Jeppefs - the "Select Region" command makes a call to determine the regions you have access to, but looks like your role might not have access to that.

Let me make one more change to read the region from your profile config. I had been putting that off because there are lots of options for how regions can be selected but it makes sense you have a profile tied to a specific region.

dacort commented 1 year ago

v0.5.1 will be published shortly that should both set your region if defined in the profile and allow you to manually select a region if the call to DescribeRegions fails. Let me know if it works for you!

Jeppefs commented 1 year ago

It works now! Thank you @dacort for all your help! this is a very nice extension now that it is up and running.