rancher / fleet

Deploy workloads from Git to large fleets of Kubernetes clusters
https://fleet.rancher.io/
Apache License 2.0
1.47k stars 216 forks source link

Deletes `Bundles` after deleting all its `BundleDeployments` #2591

Closed 0xavi0 closed 5 days ago

0xavi0 commented 5 days ago

When testing https://github.com/rancher/fleet/issues/2586 the Bundle reconciler found that the Bundle was gone as was trying to delete the same BundleDeployments in parallel.

Deleting the Bundle after deleting all its BundleDeployments seems to help to prevent that race condition between both reconcilers

Refers to: https://github.com/rancher/fleet/issues/2586

0xavi0 commented 5 days ago

Tested with the following GitRepo

apiVersion: fleet.cattle.io/v1alpha1
kind: GitRepo
metadata:
  name: sample
  namespace: fleet-local
spec:
  repo: "https://github.com/rancher/fleet-test-data"
  branch: master
  paths:
  - multiple-paths

and the following bash script:

#!/bin/bash

i=1

while true
do
  echo "---------------------------"
  kubectl apply -f test-2586.yaml

  OUT=""
  while [ "$OUT" != "2" ];
  do
   OUT=`kubectl get gitrepo sample -n fleet-local -o jsonpath='{.status.summary.ready}'`
  done

  kubectl delete gitrepo sample -n fleet-local
  kubectl wait --for=delete gitrepo/sample --timeout=60s -n fleet-local

  kubectl get bundles -A
  BUNDLES_LEFT=`kubectl get bundles -A | grep sample | wc -l`

  if [ "$BUNDLES_LEFT" == "       0" ];
  then
    echo "ALL GOOD attempt number: ${i}"
    echo "---------------------------"
    i=$(($i+1))
  else
    echo "FAILED"
    echo "---------------------------"
    exit 1
  fi
done