search

kurtosis-tech / kurtosis

A platform for packaging and launching ephemeral backend stacks with a focus on approachability for the average developer.
https://docs.kurtosistech.com/
Apache License 2.0
370 stars 53 forks source link

Adding service with same name after removing it on a k8s cluster fails #1866

Open tedim52 opened 11 months ago

tedim52 commented 11 months ago

What's your CLI version?

0.85.36

Description & steps to reproduce

  1. kurtosis service add test-enclave test2 httpd --ports http=80
  2. kurtosis service rm test-enclave test2
  3. kurtosis service add test-enclave test2 httpd --ports http=80 returns: 
    Error:  An error occurred running command 'add'
Caused by: An error occurred calling the run function for command 'add'
Caused by: An error has occurred when running Starlark to add service
Caused by: An error occurred executing instruction (number 1) at DEFAULT_PACKAGE_ID_FOR_SCRIPT[2:18]:
add_service(name="test2", config=ServiceConfig(image="httpd", ports={"http": PortSpec(number=80, wait="30s")}, files={}, entrypoint=[], cmd=[], env_vars={}, private_ip_address_placeholder="KURTOSIS_IP_ADDR_PLACEHOLDER"))
Caused by: Unexpected error occurred starting service 'test2'
Caused by: Failed registering service with name: 'test2'
Caused by: Error registering service 'test2'
Caused by: An error occurred creating Kubernetes service in enclave '29ed5c3747464c77b14e220558541658' with ID 'test2'
Caused by: Failed to create service 'test2' in namespace 'kt-test-enclave'
Caused by: services "test2" already exists

Desired behavior

The service with the same name should be able to be added, in accord with functionality over Docker cluster.

What is the severity of this bug?

Painful; this is causing significant friction in my workflow.

What area of the product does this pertain to?

CLI: the Command Line Interface

leoporoli commented 11 months ago

@Tedi it seems that the k8s service resource is not removed when you remove the enclave's service, I don't remember well but I think it was left there for a reason, or probably it's a bug. You could double-check my assumption. @mieubrisse Do you remember why the k8s service is not removed in this circumstance?

mieubrisse commented 11 months ago

Yep, I have some context here but maybe not the solution - when we were developing k8s, waiting for k8s to actually remove things took a veryyyyyy long time. We instead implemented "tombstoning", where Kurtosis would say "hey k8s, clean this up" and k8s would do it on its own sweet time while Kurtosis proceeded. That said, my understanding is that service cleanup should still happen immediately (even if Pod descheduling didn't).

So my two thoughts are either:

  1. This is tombstoning happening (in which case we'll need to think of a solution)
  2. This is just a flat bug in Kurtosis