search

kurokobo / awx-on-k3s

An example implementation of AWX on single node K3s using AWX Operator, with easy-to-use simplified configuration with ownership of data and passwords.
MIT License
553 stars 156 forks source link

remove old containers - storage waste #386

Closed amg-web closed 53 minutes ago

amg-web commented 2 weeks ago

Environment

how to remove old unused pods?

Description

after many updates hdd space become low I used k3s crictl rmi --prune - it cleaned some space

but old pods are still in a list

root@awx:~# k3s kubectl get pods --all-namespaces
NAMESPACE      NAME                                               READY   STATUS        RESTARTS       AGE
awx            awx-migration-24.0.0-dctpd                         0/1     Completed     0              168d
awx            awx-migration-24.1.0-nwlnf                         0/1     Completed     0              147d
awx            awx-migration-24.2.0-4hcs8                         0/1     Completed     0              133d
awx            awx-migration-24.3.1-8jz95                         0/1     Completed     0              118d
awx            awx-migration-24.4.0-rcg5c                         0/1     Completed     0              93d
awx            awx-migration-24.5.0-b86rb                         0/1     Completed     0              85d
awx            awx-migration-24.6.0-sc654                         0/1     Completed     0              69d
awx            awx-migration-24.6.1-gktt9                         0/1     Completed     0              13h
awx            awx-operator-controller-manager-745b55d94b-8xp9h   2/2     Running       1 (25s ago)    13h
awx            awx-postgres-15-0                                  1/1     Running       7 (21d ago)    133d
awx            awx-task-7fc7b48d56-7wkh2                          4/4     Running       0              13h
awx            awx-web-588b89b466-jhz79                           3/3     Running       0              13h
cert-manager   cert-manager-cainjector-5fd6444f95-dmlxt           1/1     Running       0              13h
cert-manager   cert-manager-d894bbbd4-mvx72                       1/1     Running       0              13h
cert-manager   cert-manager-webhook-869674f96f-488pd              1/1     Running       0              13h
kube-system    coredns-6799fbcd5-57n7s                            1/1     Running       9 (21d ago)    189d
kube-system    helm-install-traefik-crd-p2bhd                     0/1     Completed     0              15s
kube-system    helm-install-traefik-lwjpw                         0/1     Completed     0              15s
kube-system    local-path-provisioner-6795b5f9d8-t59c5            1/1     Running       0              15s
kube-system    local-path-provisioner-6c86858495-tsc2p            1/1     Terminating   10 (21d ago)   168d
kube-system    metrics-server-54fd9b65b-w5gpb                     1/1     Running       8 (21d ago)    133d
kube-system    svclb-traefik-05fee054-dp5vj                       2/2     Running       0              14s
kube-system    traefik-7d5f6474df-bcm7h                           1/1     Running       6 (21d ago)    118d
awx@awx:~$ kubectl -n awx get awx,all,ingress,secrets,issuer
NAME                      AGE
awx.awx.ansible.com/awx   189d

NAME                                                   READY   STATUS      RESTARTS      AGE
pod/awx-migration-24.0.0-dctpd                         0/1     Completed   0             168d
pod/awx-migration-24.1.0-nwlnf                         0/1     Completed   0             147d
pod/awx-migration-24.2.0-4hcs8                         0/1     Completed   0             133d
pod/awx-migration-24.3.1-8jz95                         0/1     Completed   0             118d
pod/awx-migration-24.4.0-rcg5c                         0/1     Completed   0             93d
pod/awx-migration-24.5.0-b86rb                         0/1     Completed   0             85d
pod/awx-migration-24.6.0-sc654                         0/1     Completed   0             69d
pod/awx-migration-24.6.1-gktt9                         0/1     Completed   0             14h
pod/awx-operator-controller-manager-745b55d94b-8xp9h   2/2     Running     1 (24m ago)   14h
pod/awx-postgres-15-0                                  1/1     Running     7 (22d ago)   133d
pod/awx-task-7fc7b48d56-7wkh2                          4/4     Running     0             14h
pod/awx-web-588b89b466-jhz79                           3/3     Running     0             14h

NAME                                                      TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
service/awx-operator-controller-manager-metrics-service   ClusterIP   10.43.201.65   <none>        8443/TCP   189d
service/awx-postgres-15                                   ClusterIP   None           <none>        5432/TCP   168d
service/awx-service                                       ClusterIP   10.43.86.221   <none>        80/TCP     189d

NAME                                              READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/awx-operator-controller-manager   1/1     1            1           189d
deployment.apps/awx-task                          1/1     1            1           189d
deployment.apps/awx-web                           1/1     1            1           189d

NAME                                                         DESIRED   CURRENT   READY   AGE
replicaset.apps/awx-operator-controller-manager-589cdd869b   0         0         0       180d
replicaset.apps/awx-operator-controller-manager-59b86c6fb    0         0         0       168d
replicaset.apps/awx-operator-controller-manager-6458cd4798   0         0         0       147d
replicaset.apps/awx-operator-controller-manager-66d859dc9f   0         0         0       118d
replicaset.apps/awx-operator-controller-manager-745b55d94b   1         1         1       14h
replicaset.apps/awx-operator-controller-manager-7469875f45   0         0         0       189d
replicaset.apps/awx-operator-controller-manager-7875f768df   0         0         0       69d
replicaset.apps/awx-operator-controller-manager-7bd778dbbc   0         0         0       93d
replicaset.apps/awx-operator-controller-manager-7d497b7874   0         0         0       85d
replicaset.apps/awx-operator-controller-manager-9874d5cfc    0         0         0       133d
replicaset.apps/awx-task-57ddbdf689                          0         0         0       168d
replicaset.apps/awx-task-68889c684d                          0         0         0       93d
replicaset.apps/awx-task-6968cbc89                           0         0         0       118d
replicaset.apps/awx-task-6b998cd649                          0         0         0       189d
replicaset.apps/awx-task-6dbfb7d946                          0         0         0       85d
replicaset.apps/awx-task-6ff56fb6bf                          0         0         0       133d
replicaset.apps/awx-task-7767b6c657                          0         0         0       180d
replicaset.apps/awx-task-7d4c4bdb5                           0         0         0       147d
replicaset.apps/awx-task-7fc7b48d56                          1         1         1       14h
replicaset.apps/awx-task-f9644bbf9                           0         0         0       69d
replicaset.apps/awx-web-549f9f88f5                           0         0         0       189d
replicaset.apps/awx-web-588b89b466                           1         1         1       14h
replicaset.apps/awx-web-5db56bf6b6                           0         0         0       69d
replicaset.apps/awx-web-677486c9                             0         0         0       118d
replicaset.apps/awx-web-695f968f48                           0         0         0       93d
replicaset.apps/awx-web-6bcbd5c8ff                           0         0         0       180d
replicaset.apps/awx-web-74f497f79                            0         0         0       133d
replicaset.apps/awx-web-7c59b86d44                           0         0         0       85d
replicaset.apps/awx-web-c6bdf6549                            0         0         0       147d
replicaset.apps/awx-web-d8848656c                            0         0         0       168d

NAME                               READY   AGE
statefulset.apps/awx-postgres-15   1/1     168d

NAME                             STATUS     COMPLETIONS   DURATION   AGE
job.batch/awx-migration-24.0.0   Complete   1/1           8s         168d
job.batch/awx-migration-24.1.0   Complete   1/1           7s         147d
job.batch/awx-migration-24.2.0   Complete   1/1           7s         133d
job.batch/awx-migration-24.3.1   Complete   1/1           15s        118d
job.batch/awx-migration-24.4.0   Complete   1/1           7s         93d
job.batch/awx-migration-24.5.0   Complete   1/1           8s         85d
job.batch/awx-migration-24.6.0   Complete   1/1           12s        69d
job.batch/awx-migration-24.6.1   Complete   1/1           8s         14h

NAME                                    CLASS     HOSTS                    ADDRESS           PORTS     AGE
ingress.networking.k8s.io/awx-ingress   traefik   awx.webhostingzone.org   162.247.152.240   80, 443   189d

NAME                                  TYPE                DATA   AGE
secret/awx-admin-password             Opaque              1      189d
secret/awx-app-credentials            Opaque              3      189d
secret/awx-broadcast-websocket        Opaque              1      189d
secret/awx-issuer-account-key         Opaque              1      189d
secret/awx-postgres-configuration     Opaque              6      168d
secret/awx-receptor-ca                kubernetes.io/tls   2      189d
secret/awx-receptor-work-signing      Opaque              2      189d
secret/awx-secret-key                 Opaque              1      189d
secret/awx-secret-tls                 kubernetes.io/tls   2      189d
secret/redhat-operators-pull-secret   Opaque              1      189d

NAME                                READY   AGE
issuer.cert-manager.io/awx-issuer   True    189d
kurokobo commented 2 weeks ago

@amg-web Hi, you can safely remove all jobs which can be found by kubectl -n awx get job, and all replicasets which has DESIRED=0.

However, it is unlikely that these are using a lot of disk space. If removing them do not increase the free space, you may want to use the du command something like du -sh /* | sort -hr to identify the directory which consumes a lot of disk space.

github-actions[bot] commented 5 days ago

This issue is stale because it has been open 10 days with no activity. Remove stale label or comment or this will be closed in 4 days.

github-actions[bot] commented 53 minutes ago

This issue was closed because it has been open 2 weeks with no activity.