karmada-io / karmada

Open, Multi-Cloud, Multi-Cluster Kubernetes Orchestration
https://karmada.io
Apache License 2.0
4.14k stars 813 forks source link

Dangling images left after each run of hack/local-up-karmada.sh #4914

Closed B1F030 closed 6 days ago

B1F030 commented 1 week ago

What would you like to be added: Clean up dangling images after each run of hack/local-up-karmada.sh.

Why is this needed: When we use hack/local-up-karmada.sh to start a demo environment for more than one times, there will be a lot of dangling images like this:

[root@karmada ~]# docker images
REPOSITORY                                    TAG       IMAGE ID       CREATED             SIZE
karmada/karmada-metrics-adapter               latest    857129c60dd0   10 minutes ago      96.3MB
karmada/karmada-operator                      latest    8c6fe127ba18   11 minutes ago      69.3MB
karmada/karmada-search                        latest    ab3b5473826b   11 minutes ago      106MB
karmada/karmada-interpreter-webhook-example   latest    4f60a5788642   11 minutes ago      64.3MB
karmada/karmada-scheduler-estimator           latest    6550f2107b94   12 minutes ago      72.7MB
karmada/karmada-agent                         latest    734e4aaf1282   12 minutes ago      90.7MB
karmada/karmada-webhook                       latest    fb317106d555   12 minutes ago      70.6MB
karmada/karmada-descheduler                   latest    acfd062278f0   13 minutes ago      71.4MB
karmada/karmada-scheduler                     latest    d22f16992c48   13 minutes ago      71.9MB
karmada/karmada-controller-manager            latest    b7ac5d21c076   14 minutes ago      98.6MB
karmada/karmada-aggregated-apiserver          latest    93d7db7c82c7   14 minutes ago      98.2MB
<none>                                        <none>    13db4b55023e   58 minutes ago      96.3MB
<none>                                        <none>    e55648067bab   59 minutes ago      69.3MB
<none>                                        <none>    e867d00bd410   59 minutes ago      106MB
<none>                                        <none>    19f4da333ded   About an hour ago   64.3MB
<none>                                        <none>    d9a3860f97d0   About an hour ago   72.7MB
<none>                                        <none>    cfe270c14e36   About an hour ago   90.7MB
<none>                                        <none>    7d6a215e9f90   About an hour ago   70.6MB
<none>                                        <none>    4bde55fedc46   About an hour ago   71.4MB
<none>                                        <none>    003557f7a41e   About an hour ago   71.9MB
<none>                                        <none>    8428c9530a74   About an hour ago   98.6MB
<none>                                        <none>    d7207b9f233d   About an hour ago   98.2MB
kindest/node                                  v1.27.3   89e7dc9f9131   10 months ago       932MB

Could be resolved by adding a command in script like this: docker image prune -f.

XiShanYongYe-Chang commented 1 week ago

Hi @B1F030, would you like to contribute a PR?

RainbowMango commented 1 week ago

I wonder how these dangling images were created. Is there a way to avoid them? docker image prune will remove all dangling images even those not created by Karmada. Is this the desired behavior?

zhzhuang-zju commented 1 week ago

refer to https://docs.docker.com/config/pruning/ The docker image prune command allows you to clean up unused images. By default, docker image prune only cleans up dangling images. A dangling image is one that isn't tagged, and isn't referenced by any container. Thus docker image prune -f is valid and will only clean up dangling images

how these dangling images were created

The common ways are:

B1F030 commented 1 week ago

Hi @B1F030, would you like to contribute a PR?

Sure.

I wonder how these dangling images were created. Is there a way to avoid them?

These images are generated because every time we run the hack/local-up-karmada.sh, in #step2. make images, the command make images will be triggered, refer to hack/docker.sh, and that's necessary because we need to make sure the source will be updated in each run. Two ways to resolve this problem:

  1. Use exist images instead of building new one, but as I mentioned above, It's necessary to make sure the image is updated, just like we need to set controller-manager imagePullPolicy to Always to sync with the latest version in repository.
  2. Clean up dangling images after each run.

docker image prune will remove all dangling images even those not created by Karmada. Is this the desired behavior?

Generally, dangling images have lost their value of existence and can be deleted at will. They are generated because:

  1. Errors happen during image building, lead to generate none labeled images.
  2. When building images manually, if don't commit or tag them, that will leave garbage images.
  3. These imagess take up a lot of storage space and need to be deleted.

But on the other hand, keeping dangling images could help to accelerate image building by using cache layer.