undistro / zora

Zora is an open source solution that helps you achieve compliance with Kubernetes best practices recommended by industry-leading frameworks. By scanning your cluster with multiple plugins, Zora identifies potential issues, misconfigurations, and vulnerabilities.
https://getup.io/opensource/zora-oss
Apache License 2.0
280 stars 24 forks source link

Unexpected behavior when changing the plugin of a `ClusterScan` #199

Closed matheusfm closed 1 year ago

matheusfm commented 1 year ago

Description

It's a very specific bug that happens when we change the plugin of a ClusterScan.

How to Reproduce

  1. Create a Cluster and a ClusterScan (popeye plugin by default):
    cat << EOF | kubectl apply -f -
    apiVersion: zora.undistro.io/v1alpha1
    kind: ClusterScan
    metadata:
    name: foo
    spec:
    clusterRef:
    name: foo
    schedule: "* * * * *"
    EOF
  2. Wait for a successful scan (19 issues):
    NAME   CLUSTER   SCHEDULE    SUSPEND   PLUGINS   LAST STATUS   LAST SCHEDULE   LAST SUCCESSFUL   ISSUES   READY   SAAS   AGE
    foo    foo       * * * * *   false     popeye    Complete      53s             44s               19       True    OK     26m
  3. Create a plugin that always fails:
    cat << EOF | kubectl apply -f -
    apiVersion: zora.undistro.io/v1alpha1
    kind: Plugin
    metadata:
    name: brutus
    namespace: zora-system
    spec:
    command:
    - /bin/sh
    - -c
    - |
      echo "ERROR" > \$(DONE_DIR)/error
      exit 27
    image: docker.io/derailed/popeye:v0.11.1
    EOF
  4. Update the ClusterScan to use this plugin:
    cat << EOF | kubectl apply -f -
    apiVersion: zora.undistro.io/v1alpha1
    kind: ClusterScan
    metadata:
    name: foo
    spec:
    plugins:
    - name: brutus
    clusterRef:
    name: foo
    schedule: "* * * * *"
    EOF
  5. The change does not reflect on ClusterScan that keep using popeye
    
    NAME                               CLUSTER   SCHEDULE    SUSPEND   PLUGINS   LAST STATUS   LAST SCHEDULE   LAST SUCCESSFUL   ISSUES   READY   SAAS   AGE   NEXT SCHEDULE
    clusterscan.zora.undistro.io/foo   foo       * * * * *   false     popeye    Complete      7m56s           7m47s             19       True    OK     39m   2023-03-07T13:56:00Z

NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE CONTAINERS IMAGES SELECTOR cronjob.batch/foo-popeye * False 0 56s 39m worker,popeye ghcr.io/undistro/zora/worker:v0.4.4,docker.io/derailed/popeye:v0.11.1

NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES pod/foo-popeye-27969960-t86sn 0/2 Completed 0 2m56s 10.244.0.39 kind-control-plane pod/foo-popeye-27969961-d6qx4 0/2 Completed 0 116s 10.244.0.40 kind-control-plane pod/foo-popeye-27969962-lfctl 0/2 Completed 0 56s 10.244.0.41 kind-control-plane



## Expected Behavior
The expected behavior is that the `ClusterScan` uses the new plugin.

## System Information
 - Kubernetes v1.25.3
 - Zora 0.4.4