Optimize Docker image

[mayankchhabra]

Closes #39

[lukechilds]

Did you say you thought you'd made the image bigger?

How are you checking the size, it looks smaller to me.

I just built from master and then this branch:

$ docker images umbrel-middleware-size-test                                                                                                                                              [HEAD (no branch)]
REPOSITORY                    TAG                 IMAGE ID            CREATED              SIZE
umbrel-middleware-size-test   mayank              88dd0af0cc0f        About a minute ago   248MB
umbrel-middleware-size-test   master              9912b4aae2cf        5 minutes ago        331MB

[mayankchhabra]

Yeah, I probably checked it against a cached image or something. Tested again and I've got the same resulting image size as you do.

[mayankchhabra]

Update: No, it is not the same size lol. Getting a consistent 518mb size across local Docker build, local Docker build with buildx, and mayankchhabra/middleware:patch-docker (built by GH Action using buildx)

$ docker images -a | grep middleware
mayankchhabra/middleware   local-build           e4d63277a99c        2 minutes ago       518MB
mayankchhabra/middleware   local-buildx          c1925114c08f        19 minutes ago      518MB
mayankchhabra/middleware   patch-docker          483f561d495d        33 hours ago        518MB
getumbrel/middleware       latest                3c60bb9996a0        2 weeks ago         331MB

[lukechilds]

Update: No, it is not the same size lol. Getting a consistent 518mb size across local Docker build, local Docker build with buildx, and mayankchhabra/middleware:patch-docker (built by GH Action using buildx)

$ docker images -a | grep middleware
mayankchhabra/middleware   local-build           e4d63277a99c        2 minutes ago       518MB
mayankchhabra/middleware   local-buildx          c1925114c08f        19 minutes ago      518MB
mayankchhabra/middleware   patch-docker          483f561d495d        33 hours ago        518MB
getumbrel/middleware       latest                3c60bb9996a0        2 weeks ago         331MB

To clarify you're doing:

docker build -t mayankchhabra/middleware:local-build .

in the git repo at commit 334d378?

Are you sure you don't have fluff left over from a previous install in node_modules or something?

Can you try git clone into /tmp and build from there to make sure?

Try this:

$ mkdir -p /tmp/docker-test && cd /tmp/docker-test && git clone https://github.com/mayankchhabra/umbrel-middleware.git . && git checkout "patch/docker" && docker build -t mayankchhabra/middleware:local-clean . && docker images mayankchhabra/middleware:local-clean

Cloning into '.'...
remote: Enumerating objects: 17, done.
remote: Counting objects: 100% (17/17), done.
remote: Compressing objects: 100% (13/13), done.
remote: Total 618 (delta 9), reused 12 (delta 4), pack-reused 601
Receiving objects: 100% (618/618), 1.54 MiB | 488.00 KiB/s, done.
Resolving deltas: 100% (279/279), done.
Branch 'patch/docker' set up to track remote branch 'patch/docker' from 'origin'.
Switched to a new branch 'patch/docker'
[+] Building 0.2s (13/13) FINISHED
 => [internal] load .dockerignore                                                                                                                                                                      0.0s
 => => transferring context: 276B                                                                                                                                                                      0.0s
 => [internal] load build definition from Dockerfile                                                                                                                                                   0.0s
 => => transferring dockerfile: 717B                                                                                                                                                                   0.0s
 => [internal] load metadata for docker.io/library/node:12.16.3-buster-slim                                                                                                                            0.0s
 => [umbrel-middleware-builder 1/6] FROM docker.io/library/node:12.16.3-buster-slim                                                                                                                    0.0s
 => [internal] load build context                                                                                                                                                                      0.1s
 => => transferring context: 3.98MB                                                                                                                                                                    0.1s
 => CACHED [umbrel-middleware-builder 2/6] RUN apt-get update   && apt-get install -y build-essential   && apt-get install -y python3                                                                  0.0s
 => CACHED [umbrel-middleware-builder 3/6] WORKDIR /app                                                                                                                                                0.0s
 => CACHED [umbrel-middleware-builder 4/6] COPY package*.json ./                                                                                                                                       0.0s
 => CACHED [umbrel-middleware-builder 5/6] RUN npm install --only=production                                                                                                                           0.0s
 => CACHED [umbrel-middleware-builder 6/6] COPY . .                                                                                                                                                    0.0s
 => CACHED [umbrel-middleware 2/3] COPY --from=umbrel-middleware-builder /app /app                                                                                                                     0.0s
 => CACHED [umbrel-middleware 3/3] WORKDIR /app                                                                                                                                                        0.0s
 => exporting to image                                                                                                                                                                                 0.0s
 => => exporting layers                                                                                                                                                                                0.0s
 => => writing image sha256:88dd0af0cc0fd1185df201d3092a8694bdfaed87e65e9bb0e103eaf45bf519e1                                                                                                           0.0s
 => => naming to docker.io/mayankchhabra/middleware:local-clean                                                                                                                                        0.0s
REPOSITORY                 TAG                 IMAGE ID            CREATED             SIZE
mayankchhabra/middleware   local-clean         88dd0af0cc0f        6 hours ago         248MB

[lukechilds]

Oh wow, ok it was me who had the cache issue, I tried running the above with --no-cache:

$ mkdir -p /tmp/docker-test && cd /tmp/docker-test && git clone https://github.com/mayankchhabra/umbrel-middleware.git . && git checkout "patch/docker" && docker build -t mayankchhabra/middleware:local-clean --no-cache . && docker images mayankchhabra/middleware:local-clean

Cloning into '.'...
remote: Enumerating objects: 17, done.
remote: Counting objects: 100% (17/17), done.
remote: Compressing objects: 100% (13/13), done.
remote: Total 618 (delta 9), reused 12 (delta 4), pack-reused 601
Receiving objects: 100% (618/618), 1.54 MiB | 474.00 KiB/s, done.
Resolving deltas: 100% (279/279), done.
Branch 'patch/docker' set up to track remote branch 'patch/docker' from 'origin'.
Switched to a new branch 'patch/docker'
[+] Building 328.0s (13/13) FINISHED
 => [internal] load build definition from Dockerfile                                                                                                                                                   0.0s
 => => transferring dockerfile: 717B                                                                                                                                                                   0.0s
 => [internal] load .dockerignore                                                                                                                                                                      0.0s
 => => transferring context: 276B                                                                                                                                                                      0.0s
 => [internal] load metadata for docker.io/library/node:12.16.3-buster-slim                                                                                                                            0.0s
 => [internal] load build context                                                                                                                                                                      0.1s
 => => transferring context: 3.98MB                                                                                                                                                                    0.1s
 => CACHED [umbrel-middleware-builder 1/6] FROM docker.io/library/node:12.16.3-buster-slim                                                                                                             0.0s
 => [umbrel-middleware-builder 2/6] RUN apt-get update   && apt-get install -y build-essential   && apt-get install -y python3                                                                        84.6s
 => [umbrel-middleware-builder 3/6] WORKDIR /app                                                                                                                                                       0.0s
 => [umbrel-middleware-builder 4/6] COPY package*.json ./                                                                                                                                              0.0s
 => [umbrel-middleware-builder 5/6] RUN npm install --only=production                                                                                                                                233.2s
 => [umbrel-middleware-builder 6/6] COPY . .                                                                                                                                                           0.1s
 => [umbrel-middleware 2/3] COPY --from=umbrel-middleware-builder /app /app                                                                                                                            4.5s
 => [umbrel-middleware 3/3] WORKDIR /app                                                                                                                                                               0.0s
 => exporting to image                                                                                                                                                                                 3.0s
 => => exporting layers                                                                                                                                                                                3.0s
 => => writing image sha256:6f2c8eb219071bb0606c30c348cb762515ff0c62b3d5405a1c1741bd2e212fc4                                                                                                           0.0s
 => => naming to docker.io/mayankchhabra/middleware:local-clean                                                                                                                                        0.0s
REPOSITORY                 TAG                 IMAGE ID            CREATED             SIZE
mayankchhabra/middleware   local-clean         6f2c8eb21907        3 seconds ago       518MB

[mayankchhabra]

@lukechilds Spot on! I did a clean run with the one-liner you wrote and was able to replicate the same results. Thanks for that!

REPOSITORY                 TAG                 IMAGE ID            CREATED                  SIZE
mayankchhabra/middleware   local-clean         9c8b65b75df9        Less than a second ago   248MB

[mayankchhabra]

Lol then where did my cache go? 😂

Ok, trying with --no-cache now...

[mayankchhabra]

🤦‍♂️

mkdir -p /tmp/docker-test && cd /tmp/docker-test && git clone https://github.com/mayankchhabra/umbrel-middleware.git . && git checkout "patch/docker" && docker build -t mayankchhabra/middleware:local-clean --no-cache . && docker images mayankchhabra/middleware:local-clean
Cloning into '.'...
remote: Enumerating objects: 20, done.
remote: Counting objects: 100% (20/20), done.
remote: Compressing objects: 100% (14/14), done.
remote: Total 621 (delta 11), reused 15 (delta 6), pack-reused 601
Receiving objects: 100% (621/621), 1.54 MiB | 736.00 KiB/s, done.
Resolving deltas: 100% (281/281), done.
Checking connectivity... done.
Branch patch/docker set up to track remote branch patch/docker from origin.
Switched to a new branch 'patch/docker'
Sending build context to Docker daemon  4.016MB
Step 1/11 : FROM node:12.16.3-buster-slim AS umbrel-middleware-builder
 ---> b3d2a7266fed
Step 2/11 : RUN apt-get update   && apt-get install -y build-essential   && apt-get install -y python3
 ---> Running in 9aea5fa13b6f
Get:1 http://security.debian.org/debian-security buster/updates InRelease [65.4 kB]
Get:2 http://security.debian.org/debian-security buster/updates/main amd64 Packages [213 kB]
Get:3 http://deb.debian.org/debian buster InRelease [121 kB]
Get:4 http://deb.debian.org/debian buster-updates InRelease [51.9 kB]
Get:5 http://deb.debian.org/debian buster/main amd64 Packages [7905 kB]
Get:6 http://deb.debian.org/debian buster-updates/main amd64 Packages [7868 B]
Fetched 8365 kB in 13s (628 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
...
Removing intermediate container 9aea5fa13b6f
 ---> 4c4fe721eb08
Step 3/11 : WORKDIR /app
 ---> Running in 2835b9d6a2f1
Removing intermediate container 2835b9d6a2f1
 ---> 10750574ab1f
Step 4/11 : COPY package.json package-lock.json ./
 ---> b8fb8c48b500
Step 5/11 : RUN npm install --only=production
 ---> Running in 51e787dca45a

> grpc@1.24.2 install /app/node_modules/grpc
> node-pre-gyp install --fallback-to-build --library=static_library

node-pre-gyp WARN Using needle for node-pre-gyp https download 
[grpc] Success: "/app/node_modules/grpc/src/node/extension_binary/node-v72-linux-x64-glibc/grpc_node.node" is installed via remote
npm WARN umbrel-middleware@0.1.2 No repository field.
npm WARN umbrel-middleware@0.1.2 No license field.

added 679 packages from 1014 contributors and audited 1047 packages in 45.486s

2 packages are looking for funding
  run `npm fund` for details

found 16 vulnerabilities (10 low, 1 moderate, 4 high, 1 critical)
  run `npm audit fix` to fix them, or `npm audit` for details
Removing intermediate container 51e787dca45a
 ---> b5316b5e8e5d
Step 6/11 : COPY . .
 ---> 47e2423dd565
Step 7/11 : FROM node:12.16.3-buster-slim AS umbrel-middleware
 ---> b3d2a7266fed
Step 8/11 : COPY --from=umbrel-middleware-builder /app /app
 ---> a637459a9d45
Step 9/11 : WORKDIR /app
 ---> Running in bbe7ddc5ae7f
Removing intermediate container bbe7ddc5ae7f
 ---> b9211e17e556
Step 10/11 : EXPOSE 3005
 ---> Running in 81070a46170e
Removing intermediate container 81070a46170e
 ---> 8f9944c381e7
Step 11/11 : CMD [ "npm", "start" ]
 ---> Running in 8b9367abbdda
Removing intermediate container 8b9367abbdda
 ---> 9f1725321af0
Successfully built 9f1725321af0
Successfully tagged mayankchhabra/middleware:local-clean
REPOSITORY                 TAG                 IMAGE ID            CREATED                  SIZE
mayankchhabra/middleware   local-clean         9f1725321af0        Less than a second ago   248MB

[mayankchhabra]

Enough is enough: https://github.com/getumbrel/umbrel-middleware/pull/46

[mayankchhabra]

Closed by #46