Can we re-build the Envoy Docker image without Internet

[YvesZHI]

I'm trying to build an Envoy Docker image on my Ubuntu 18.04. I just followed the doc: https://www.envoyproxy.io/docs/envoy/latest/start/building/local_docker_build, which means that I executed the two commands as below:

sudo -E ./ci/run_envoy_docker.sh './ci/do_ci.sh release.server_only'
ENVOY_DOCKER_IN_DOCKER=1 sudo -E ./ci/run_envoy_docker.sh './ci/do_ci.sh docker'

It works as expected but I just found that if I re-execute the command ENVOY_DOCKER_IN_DOCKER=1 sudo -E ./ci/run_envoy_docker.sh './ci/do_ci.sh docker', all of necessary docker images, which have been downloaded last time, will be downloaded again.

I removed all of my local docker images and executed the command ENVOY_DOCKER_IN_DOCKER=1 sudo -E ./ci/run_envoy_docker.sh './ci/do_ci.sh docker', I found that the docker images have been downloaded after the command finishes:

envoy@my-server:~/workspace/envoy$ sudo docker images
REPOSITORY                       TAG               IMAGE ID       CREATED          SIZE
envoyproxy/envoy-tools           v1.28.0           0659097999ca   9 seconds ago    225MB
envoyproxy/envoy-google-vrp      v1.28.0           53da34e25a9b   25 seconds ago   413MB
envoyproxy/envoy-distroless      v1.28.0           4f25ab0e7885   2 minutes ago    83.7MB
envoyproxy/envoy-contrib-debug   v1.28.0           38b94a5c7085   2 minutes ago    1.17GB
envoyproxy/envoy-contrib         v1.28.0           aaaa62911e1c   3 minutes ago    189MB
envoyproxy/envoy-debug           v1.28.0           1a9d7f7e13d7   3 minutes ago    1.02GB
envoyproxy/envoy                 v1.28.0           4c079f3e2b18   3 minutes ago    165MB
moby/buildkit                    buildx-stable-1   ee33f441bff7   3 days ago       172MB
envoyproxy/envoy-build-ubuntu    <none>            cbe615809347   6 days ago       5.67GB
tonistiigi/binfmt                latest            354472a37893   14 months ago    60.2MB

However, if I now execute the command ENVOY_DOCKER_IN_DOCKER=1 sudo -E ./ci/run_envoy_docker.sh './ci/do_ci.sh docker again, the docker images will be downloaded again. Here is the output of sudo docker images:

REPOSITORY                       TAG               IMAGE ID       CREATED          SIZE
envoyproxy/envoy-tools           v1.28.0           5d3cf1daf849   29 seconds ago   225MB
envoyproxy/envoy-google-vrp      v1.28.0           d0dbe063ee58   45 seconds ago   413MB
envoyproxy/envoy-distroless      v1.28.0           d65d2c0135cc   2 minutes ago    83.7MB
envoyproxy/envoy-contrib-debug   v1.28.0           a6d6a153438c   2 minutes ago    1.17GB
envoyproxy/envoy-contrib         v1.28.0           d11e15d2787a   2 minutes ago    189MB
envoyproxy/envoy-debug           v1.28.0           f6e37f630885   3 minutes ago    1.02GB
envoyproxy/envoy                 v1.28.0           a0127ac8bfa4   3 minutes ago    165MB
<none>                           <none>            0659097999ca   4 minutes ago    225MB
<none>                           <none>            53da34e25a9b   4 minutes ago    413MB
<none>                           <none>            4f25ab0e7885   6 minutes ago    83.7MB
<none>                           <none>            38b94a5c7085   7 minutes ago    1.17GB
<none>                           <none>            aaaa62911e1c   7 minutes ago    189MB
<none>                           <none>            1a9d7f7e13d7   7 minutes ago    1.02GB
<none>                           <none>            4c079f3e2b18   8 minutes ago    165MB
moby/buildkit                    buildx-stable-1   ee33f441bff7   3 days ago       172MB
envoyproxy/envoy-build-ubuntu    <none>            cbe615809347   6 days ago       5.67GB
tonistiigi/binfmt                latest            354472a37893   14 months ago    60.2MB

As you see, all of <none> images are duplicated now.

Besides, while building the Envoy Docker image, I can see the log:

 => [internal] booting buildkit                                                                                                                                                                        5.4s
 => => pulling image moby/buildkit:buildx-stable-1                                                                                                                                                     4.3s
 => => creating container buildx_buildkit_multi-builder0                                                                                                                                               1.1s
 => [internal] load build definition from Dockerfile-envoy                                                                                                                                             0.1s
 => => transferring dockerfile: 4.06kB                                                                                                                                                                 0.0s
 => [internal] load metadata for docker.io/library/ubuntu:20.04@sha256:33a5cc25d22c45900796a1aca487ad7a7cb09f09ea00b779e3b2026b4fc2faba                                                                5.7s
 => [internal] load .dockerignore                                                                                                                                                                      0.0s
 => => transferring context: 297B                                                                                                                                                                      0.0s
 => [envoy-base 1/4] FROM docker.io/library/ubuntu:20.04@sha256:33a5cc25d22c45900796a1aca487ad7a7cb09f09ea00b779e3b2026b4fc2faba                                                                       4.7s
 => => resolve docker.io/library/ubuntu:20.04@sha256:33a5cc25d22c45900796a1aca487ad7a7cb09f09ea00b779e3b2026b4fc2faba                                                                                  0.0s
 => => sha256:edaedc954fb53f42a7754a6e2d1b57f091bc9b11063bc445c2e325ea448f8f68 27.51MB / 27.51MB                                                                                                       3.8s
 => => extracting sha256:edaedc954fb53f42a7754a6e2d1b57f091bc9b11063bc445c2e325ea448f8f68

As you see, it seems that load metadata for docker.io/library/ubuntu:20.04@sha256:..... always appear no matter how many times that the command ENVOY_DOCKER_IN_DOCKER=1 sudo -E ./ci/run_envoy_docker.sh './ci/do_ci.sh docker is executed. I don't know why can't the local image be used immediately...

In a word, I have two questions:

• Why are those docker images downloaded again and again, instead of using the local docker images immediately?
• Can I re-build Envoy Docker image (re-execute the command ENVOY_DOCKER_IN_DOCKER=1 sudo -E ./ci/run_envoy_docker.sh './ci/do_ci.sh docker) without Internet?

[alyssawilk]

cc @phlax

[phlax]

first point is that its using the host docker - so running inside the container etc is not relevant - it should be the same

wrt re-downloading the images they have different image ids - they havent been redownloaded they have been rebuilt

the images with <none> are intermediate build layers that are kept around to speedup subsequent builds

im not seeing anything unexpected above - docker will use the internet (at least by default) to check base layers but there is nothing to suggest anythign be redownloaded in logs above

its quite possible that we could improve the layering in our build (i did some work on this already) to speed up rebuilds but i think that is a separate question

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

[github-actions[bot]]

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted" or "no stalebot". Thank you for your contributions.