ibm-mas / ansible-devops

Ansible collection supporting devops for IBM Maximo Application Suite
https://ibm-mas.github.io/ansible-devops/
Eclipse Public License 2.0
49 stars 83 forks source link

Add a new Role to prune Manage images from ocp image-registry #990

Closed lokesh-sreedhara closed 1 year ago

lokesh-sreedhara commented 1 year ago

The Manage Admin-config builds uses ImageStreams to store images on ocp cluster image-registry. These images are not automatically pruned by the cluster pruner as they are referenced by a imagestream. Manage Image Streams needs to be deleted before purning the images, Manage Install on Long running cluster tend to break when image-registry is full . this also degrades the performance of the overall cluster.

Lets add a new ansible role to prune the manage build images from image-registry. here are the instructions on how to perform this cleanup.

Steps taken to clean up old manage admin-config builds from openshift-image-registry

Builds->Imagestreams Handy script to delete all imagestreams

echo ""
echo "======================================================================" 
echo "Delete image streams"
echo "======================================================================" 
mng_projects=$(oc projects | awk '{print $1}' | grep mas | grep manage)
for mng_project in ${mng_projects[@]} ; do
  echo ""
  echo "Get image streams for $mng_project namespace"
  echo "----------------------------------------------------------------------"
  img_streams=$(oc get imagestreams -n $mng_project --no-headers | awk '{print $1}')
  for img_stream in ${img_streams[@]} ; do
    echo "Delete image stream $img_stream of $mng_project namespace"
    oc delete imagestream $img_stream -n $mng_project
  done
done
oc project openshift-image-registry 
oc apply -f prune.yml

prune.yml

kind: List
apiVersion: v1
items:
- apiVersion: v1
  kind: ServiceAccount
  metadata:
    name: pruner
    namespace: openshift-image-registry
- apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRoleBinding
  metadata:
    name: openshift-image-registry-pruner
    namespace: openshift-image-registry
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: system:image-pruner
  subjects:
  - kind: ServiceAccount
    name: pruner
    namespace: openshift-image-registry
- apiVersion: batch/v1
  kind: Job
  metadata:
    name: custom-image-pruner
    namespace: openshift-image-registry
  spec:
    template:
      spec:
        containers:
        - image: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4ce010cf8a1292ce086a6c321775578aaa492c8dbb2186976c8df4c824f41615"
          resources:
            requests:
              cpu: 1
              memory: 1Gi
          terminationMessagePolicy: FallbackToLogsOnError
          command:
          - oc
          args:
          - adm
          - prune
          - images
          - --certificate-authority=/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt
          - --keep-tag-revisions=1
          - --keep-younger-than=120h
          - --ignore-invalid-refs
          - --confirm
          name: image-pruner
        serviceAccountName: pruner
        restartPolicy: Never
andrercm commented 1 year ago

@lokesh-sreedhara is this still needed?

andrercm commented 1 year ago

Closing this based off a chat with Jenny: https://ibm-watson-iot.slack.com/archives/C010DGARLDP/p1693413984689639?thread_ts=1693396480.399129&cid=C010DGARLDP