search

skupperproject / skupper

Skupper is an implementation of a Virtual Application Network, enabling rich hybrid cloud communication.
http://skupper.io
Apache License 2.0
595 stars 74 forks source link

[v2] After creating/deleting sites east/west, see many logs reporting: [ControllerError] Error while handling Certificate #1701

Closed Karen-Schoener closed 1 month ago

Karen-Schoener commented 1 month ago

Describe the bug After creating/deleting sites east/west, see many logs reporting: [ControllerError] Error while handling Certificate west/skupper-site-ca: secrets \"skupper-site-ca\" not found

Issue is consistently reproducible.

How To Reproduce

    kubectl apply  -f ./cmd/controller/deploy_cluster_scope.yaml

    kubectl apply  -n west -f cmd/controller/example/site1.yaml
    Wait 10 seconds.

    kubectl apply  -n east -f cmd/controller/example/site2.yaml
    Wait 10 seconds.

    kubectl delete -n west -f cmd/controller/example/site1.yaml
    At this point, skupper-controller issues many logs: [ControllerError] Error while handling Certificate...

Snippet of skupper-controller logs:

time=2024-10-02T21:13:17.630Z level=INFO msg="Checking Certificate west/skupper-local-server"
time=2024-10-02T21:13:17.824Z level=INFO msg="[ControllerError] Error while handling Certificate west/skupper-local-server: secrets \"skupper-local-server\" not found"
time=2024-10-02T21:13:17.829Z level=INFO msg="Checking Certificate west/skupper-local-server"
time=2024-10-02T21:13:18.027Z level=INFO msg="[ControllerError] Error while handling Certificate west/skupper-local-server: secrets \"skupper-local-server\" not found"
time=2024-10-02T21:13:18.032Z level=INFO msg="Checking Certificate west/skupper-local-server"
time=2024-10-02T21:13:18.226Z level=INFO msg="[ControllerError] Error while handling Certificate west/skupper-local-server: secrets \"skupper-local-server\" not found"
time=2024-10-02T21:13:18.232Z level=INFO msg="Checking Certificate west/skupper-local-server"
time=2024-10-02T21:13:18.427Z level=INFO msg="[ControllerError] Error while handling Certificate west/skupper-local-server: secrets \"skupper-local-server\" not found"
time=2024-10-02T21:13:18.433Z level=INFO msg="Checking Certificate west/skupper-local-server"
time=2024-10-02T21:13:18.624Z level=INFO msg="[ControllerError] Error while handling Certificate west/skupper-local-server: secrets \"skupper-local-server\" not found"

Expected behavior Quiet logs after deleting sites east/west.

Environment details

Additional context Running with skupper v2 branch at commit with date of Thursday Sept 26. 

commit 653f754e88d826b80cae340d90fbca10c297000b (HEAD -> v2, origin/v2)

My sandbox has no local changes.

Karen-Schoener commented 1 month ago

I think the issue is consistently reproducible on my VM because it takes longer than 10 seconds for both sites report as 'ok' in the kubectl get site command.

$ kubectl get site -A
NAMESPACE   NAME   STATUS                                          SITES IN NETWORK
east        east   OK                                              1
west        west   containers with unready status: [config-sync]   1

Basically, with the test steps in this ticket, I am deleting a site that is not in the OK state.

I was worried that the issue was due to some misconfig on my VM, so I re-ran the test steps and added local debug logging in pkg/kube/certificates/mgr.go.

In certificateDeleted(), I noticed m.controller.GetKubeClient().CoreV1().Secrets.Delete return: err=secrets \"skupper-site-ca\" not found"

Karen-Schoener commented 1 month ago

Verified that PR https://github.com/skupperproject/skupper/pull/1703 fixes this issue.

Closing issue.