search

kurtosis-tech / kurtosis

A platform for packaging and launching ephemeral backend stacks with a focus on approachability for the average developer.
https://docs.kurtosistech.com/
Apache License 2.0
352 stars 50 forks source link

investigate the interplay between cluster set & kubectl current-context when multiple clusters are configured #1554

Open h4ck3rk3y opened 11 months ago

h4ck3rk3y commented 11 months ago

What's your CLI version?

0.84.3

Description & steps to reproduce

config-version: 2
should-send-metrics: true
kurtosis-clusters:
  cloud:
    type: kubernetes
    config:
      kubernetes-cluster-name: arn:aws:eks:us-east-2:258623609258:cluster/gyani-cluster
      storage-class: standard
      enclave-size-in-megabytes: 10
  docker:
    type: docker
  minikube:
    type: kubernetes
    config:
      kubernetes-cluster-name: minikube
      storage-class: standard
      enclave-size-in-megabytes: 10

I have a kurtosis config that looks like ^; earlier it had a digital ocean cluster as well. The gyani-cluster has no node groups but still things are spinning up; if I stop the gateway it prints that its stopping connections with a digital ocean url; so cloud is actually launching things in the digital ocean cluster. The ./.kube/config file doesn't mix the two; the gyani-cluster has aws urls. So I am confused whats' going on.

Seems like my current-context in k8s is set to the digital ocean cluster & Kurtosis seems to go off of that instead of kurtosis cluster set. Changing current-context on the kube config is making kurtosis engine restart fail as expected (no node groups on aws)

Desired behavior

Kurtosis uses the value in kurtosis cluster get instead of the current context.

What is the severity of this bug?

Papercut; this bug is frustrating, but I have a workaround.

What area of the product does this pertain to?

CLI: the Command Line Interface

h4ck3rk3y commented 11 months ago

oh so here is the order of operations

  1. Install Kurtosis
  2. Add AWS EKS
  3. Some time passes
  4. Add Digital Ocean Kubernetes
  5. Use kurtosis cluster set digitalocean for a bit
  6. Shift to docker
  7. Some more time passes
  8. kurtosis cluster set cloud (note cloud is aws)
  9. Kurtosis still talks to digital ocean
  10. Change current-context to aws
  11. Kurtosis uses aws

I'd expect at 9 Kurtosis would talk to aws but it doesn't

mieubrisse commented 11 months ago

Potentially related: #1080