microsoft / mindaro

Bridge to Kubernetes - for Visual Studio and Visual Studio Code
MIT License
307 stars 106 forks source link

Unable to connect to EKS that requires use of aws cli to generate user token #126

Open nullin opened 3 years ago

nullin commented 3 years ago

Describe the bug Unable to get the bridge setup with kube config for an EKS cluster that specifies the use of aws command line tool. See sample kubeconfig for the EKS cluster

To Reproduce Steps to follow to reproduce this issue.

Use a kubeconfig like:

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: <a valid encoded string here>
    server: https://340A4A2B8FFA14C14BA4EB7.gr7.us-west-2.eks.amazonaws.com
  name: arn:aws:eks:us-west-2:12252432:cluster/dev-eks4
contexts:
- context:
    cluster: arn:aws:eks:us-west-2:12252432:cluster/dev-eks4
    namespace: foobar
    user: arn:aws:eks:us-west-2:12252432:cluster/dev-eks4
  name: dev-eks4
current-context: dev-eks4
kind: Config
preferences: {}
users:
- name: arn:aws:eks:us-west-2:122524323692:cluster/qa-eks4
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1alpha1
      args:
      - --region
      - us-west-2
      - eks
      - get-token
      - --cluster-name
      - dev-eks4
      command: aws
      env:
      - name: AWS_PROFILE
        value: dev
      provideClusterInfo: false

This config uses aws cli to generate a token to connect to EKS.

On starting up the bridge, you get an error:

> Executing task: bridge-to-kubernetes.service <

Redirecting Kubernetes service stem-kyber to your machine...
Target cluster: arn:aws:eks:us-west-2:12252432:cluster/dev-eks4
Current cluster: arn:aws:eks:us-west-2:12252432:cluster/dev-eks4
Target namespace: foobar
Current namespace: foobar
Target service name: test-svc
Target service ports: 80
Using kubernetes service environment variables: false

Error: Failed to load current kubeconfig. Please confirm that your kubeconfig is valid.

The terminal process terminated with exit code: 1.

Expected behavior The sample kubeconfig specified above should be supported.

All other things in setup, imho, are configured properly, because, on replacing the user config in kubeconfig by the actual token, bridge starts up fine.

Setting up some thing like:

...
...
users:
- name: arn:aws:eks:us-west-2:12252432:cluster/dev-eks4
  user:
    token: <valid-token-here>

works fine and the bridge starts up. But this workaround works only temporarily as the token expires within the hour.

Logs Attach logs from the following directory: For OSX/Linux: $TMPDIR/Bridge to Kubernetes

╰─❯ cat $TMPDIR/Bridge\ To\ Kubernetes/bridge-mindarocli-2021-03-03-19-44-48-89862.txt
2021-03-03T19:44:49.3294900Z | MindaroCli | TRACE | Event: Command.Start <json>{"properties":{"arguments":"prep-connect --output json","isRoutingEnabled":"false"},"metrics":null}</json>\nOperation context: <json>{"clientRequestId":null,"correlationId":"1d5de705-3564-4df2-9148-9f93e3ce36901614761606953:3a3d37ad51ec","requestId":null,"userSubscriptionId":null,"startTime":"2021-03-03T19:44:49.0151600Z","userAgent":"VSCode/1.0.120210224","requestHttpMethod":null,"requestUri":null,"version":"1.0.20210224.1","requestHeaders":{},"loggingProperties":{"applicationName":"MindaroCli","deviceOperatingSystem":"Darwin 20.3.0 Darwin Kernel Version 20.3.0: Thu Jan 21 00:07:06 PST 2021; root:xnu-7195.81.3~1/RELEASE_X86_64","framework":".NET Core 3.1.9","macAddressHash":"301cc8cdf14fd7264abe37999c3fb57005bd289fac1169719e1bc0ecf36a506d","processId":89862,"targetEnvironment":"Production","commandId":"3a3d37ad51ec"}}</json>
2021-03-03T19:44:49.3769170Z | MindaroCli | TRACE | Running Microsoft.BridgeToKubernetes.Exe.Commands.Connect.PrepConnectCommand...
2021-03-03T19:44:49.3786310Z | MindaroCli | TRACE | Initializing progress reporter...
2021-03-03T19:44:49.5062790Z | MindaroCli | WARNG | Failed to load current kubeconfig. Please confirm that your kubeconfig is valid.
2021-03-03T19:44:49.5073690Z | MindaroCli | ERROR | Failed to load current kubeconfig. Please confirm that your kubeconfig is valid.\n
2021-03-03T19:44:49.5176200Z | MindaroCli | TRACE | Event: Command.End <json>{"properties":{"arguments":"prep-connect --output json","result":"Failed","failureReason":"Failed to load current kubeconfig. Please confirm that your kubeconfig is valid."},"metrics":{"duration":243.0}}</json>

Environment Details Client used (VS Code):

╰─❯ code --list-extensions --show-versions
Cameron.vscode-pytest@0.1.1
Equinusocio.vsc-community-material-theme@1.4.2
Equinusocio.vsc-material-theme@33.1.2
equinusocio.vsc-material-theme-icons@1.2.2
mindaro-dev.file-downloader@1.0.10
mindaro.mindaro@1.0.120210224
ms-kubernetes-tools.vscode-kubernetes-tools@1.3.0
ms-python.python@2021.2.582707922
ms-python.vscode-pylance@2021.2.4
ms-toolsai.jupyter@2021.2.603412351
redhat.vscode-yaml@0.15.0
VisualStudioExptTeam.vscodeintellicode@1.2.11

╰─❯ code --version
1.53.2
622cb03f7e070a9670c94bae1a45d78d7181fbd4
x64

Client's version: 1.53.2 Operating System: MacOS

Additional context We also have AWS SSO configured.

amsoedal commented 3 years ago

Hi @nullin, thanks for reporting this issue. We've been seeing this error in connection with some other kubeconfig configurations as well, but I hadn't seen this one before. I will bring this up to the team and update this thread when I have more info to share. Thanks for your patience!

nullin commented 3 years ago

Hi, any updated timeline on when you'll be able to get to this? Thanks.

amsoedal commented 3 years ago

Hi @nullin, we have updated the package that we rely on to parse the kubeconfig. Can you try again with the latest release and let me know if it works for you?