search

balena-io-modules / balena-preload

Script for preloading containers onto balena device images
https://www.balena.io/
Apache License 2.0
35 stars 8 forks source link

Use custom dind image based on alpine #251

Closed klutchell closed 3 years ago

klutchell commented 3 years ago

Support additional platforms like armv7 by building our own DinD image.

brainstorm discussions

klutchell commented 3 years ago

Tested on my workstation:

Linux system76 5.11.0-7620-generic #21~1624379747~21.04~3abeff8-Ubuntu SMP Wed Jun 23 02:34:03 UTC  x86_64 x86_64 x86_64 GNU/Linux
Client: Docker Engine - Community
 Version:           20.10.7
 API version:       1.41
 Go version:        go1.13.15
 Git commit:        f0df350
 Built:             Wed Jun  2 11:56:41 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.7
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       b0f5bc3
  Built:            Wed Jun  2 11:54:53 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.6
  GitCommit:        d71fcd7d8303cbf684402823e425e9dd2e99285d
 runc:
  Version:          1.0.0-rc95
  GitCommit:        b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
tmigone commented 3 years ago

@klutchell I just tested this PR combined with #252 it and it worked great, details on my workstation below in case they help. I noticed however that the final compressed image size while smaller is not that big of a difference, it's down to 96MB from 103MB so ~7-8%. I'm not complaining, i'll take any improvement we can find but since you mentioned we are not using most of the official docker tools I wonder why the difference is "so small"?

Maybe @rahul-thakoor can help us further reduce the image? :D

ubuntu@balena:~/$ uname -a
Linux balena 5.4.0-77-generic #86-Ubuntu SMP Thu Jun 17 02:35:03 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

ubuntu@balena:~/$ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 1
 Server Version: 20.10.7
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: d71fcd7d8303cbf684402823e425e9dd2e99285d
 runc version: b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 5.4.0-77-generic
 Operating System: Ubuntu 20.04.2 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 1
 Total Memory: 3.844GiB
 Name: balena
 ID: G4LX:JHQA:LF4Y:J447:ZV6G:V5KJ:ZHJS:IVGX:TPIY:QUQA:OCBX:XLPZ
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: tmigone
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No swap limit support
klutchell commented 3 years ago

@tmigone Thanks for helping to test! Most of the space is the docker binaries so we can't reduce it much further.

The one potential benefit this brings is armv6 and armv7 support since docker stopped publishing those images at v19.03

But do we even want/need to support preload on 32-bit ARM devices? I know there is a use case for testbot, and some users are trying it, but that doesn't mean we need to support it when the official images don't. Will probably end up on a product call at some point.

See this discussion: https://github.com/balena-io/balena-cli/issues/2290#issuecomment-875948020

klutchell commented 3 years ago

@balena-ci retest

klutchell commented 3 years ago

@balena-ci retest

klutchell commented 3 years ago

@balena-ci I self-certify!