search

ChristopherHX / gitea-actions-runner

MIT License
9 stars 2 forks source link

Add Support for Docker-in-Podman #12

Open oilrich25 opened 8 months ago

oilrich25 commented 8 months ago

Background

The current use of the docker:dind-rootless solution has some limitations in certain scenarios. In order to enhance the flexibility of container management, we have decided to replace it with quay.io/containers/podman to support more customizable configurations and container image management capabilities.

Changes Made

We have replaced the original docker:dind-rootless with quay.io/containers/podman. The advantage of this change is the ability to specify a private registry, avoiding potential delays in image pulling due to network speed and connection issues. To achieve this, we added support for registries.conf in the container, allowing users to customize the registries configuration for more flexible image management.

Advantages of Using Podman

Compared to Docker, Podman provides more robust container management features, especially regarding registries. Here's an example showing how to configure registries.conf to specify different container registries:

unqualified-search-registries = ['docker.io', 'k8s.gcr.io', 'gcr.io', 'ghcr.io', 'quay.io']

[[registry]]
prefix = "docker.io"
insecure = true
location = "192.168.31.102:8082"

[[registry]]
prefix = "k8s.gcr.io"
insecure = true
location = "192.168.31.102:8082"

[[registry]]
prefix = "gcr.io"
insecure = true
location = "192.168.31.102:8082"

[[registry]]
prefix = "ghcr.io"
insecure = true
location = "192.168.31.102:8082"

[[registry]]
prefix = "quay.io"
insecure = true
location = "192.168.31.102:8082"

[[registry]]
prefix = "registry.semaphoreci.com"
insecure = true
location = "127.0.0.1:8082"

[[registry]]
location="192.168.31.108:80"
insecure=true

[[registry]]
location="192.168.31.102:8083"
insecure=true

This configuration allows flexible management of images from various container registries.

Docker-Compose Support

For convenience, we provide a simple docker-compose.yaml file that supports both podman-compose and docker-compose. This configuration has been tested and is ready for use. Please note that DOCKER_HOST has been updated to support Podman:

# docker-compose.yaml
version: '3'
networks:
  runner:
    external: false
volumes:
  runner:
    driver: local
  runner-externals:
    driver: local
  gitea-runner-data:
    driver: local
services:
  runner:
    image: ghcr.io/christopherhx/gitea-actions-runner:latest
    environment:
      - GITEA_INSTANCE_URL=https://gitea.com
      - GITEA_RUNNER_REGISTRATION_TOKEN=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
      - GITEA_RUNNER_LABELS=self-hosted
      - DOCKER_HOST=tcp://podman:2376  # Updated to support Podman
    restart: always
    networks:
      - runner
    volumes:
      - gitea-runner-data:/data
      - runner:/home/runner/_work
      - runner-externals:/home/runner/externals
    depends_on:
      - podman
  podman:
    image: quay.io/containers/podman:latest
    restart: always
    privileged: true
    command: "podman system service --log-level=debug --time=0 tcp://podman:2376"
    networks:
      - runner
    volumes:
      - runner:/home/runner/_work
      - runner-externals:/home/runner/externals
      - ./containers/registries.conf:/etc/containers/registries.conf

Feel free to add this configuration to the ChristopherHX/gitea-actions-runner repository and test it. If you have any issues or suggestions, please don't hesitate to bring them up. Thank you for your assistance!

oilrich25 commented 8 months ago

docker exec -it oilrich25-runner-1 docker info

[root@ouchiharbor oilrich25]# docker exec -it oilrich25-runner-1 docker info
Client:
 Version:    24.0.6
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2
    Path:     /usr/local/lib/docker/cli-plugins/docker-buildx

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 1
 Server Version: 4.8.1
 Storage Driver: overlay
  Backing Filesystem: xfs
  Supports d_type: true
  Native Overlay Diff: false
  Using metacopy: false
  Supports shifting: true
  Supports volatile: true
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge macvlan ipvlan
  Log: k8s-file none passthrough journald
 Swarm: inactive
 Runtimes: crun crun-wasm youki kata krun ocijail runc runj runsc
 Default Runtime: crun
 Init Binary: 
 containerd version: 
 runc version: 
 init version: 
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 4.14.294-220.533.amzn2.x86_64
 Operating System: fedora
 OSType: linux
 Architecture: amd64
 CPUs: 20
 Total Memory: 23.54GiB
 Name: e06a03a5fbc2
 ID: 801d4fbd-fdd5-4131-a653-e9902b3fce23
 Docker Root Dir: /var/lib/containers/storage
 Debug Mode: true
  File Descriptors: 10
  Goroutines: 16
  System Time: 2023-12-15T03:23:01.66817842Z
  EventsListeners: 0
 Experimental: true
 Insecure Registries:
  docker.io
  gcr.io
  ghcr.io
  k8s.gcr.io
  quay.io
 Live Restore Enabled: false
 Product License: Apache-2.0

WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
oilrich25 commented 8 months ago

Of course, Docker-in-Podman only provides an option. I still strongly retain the original Docker-in-Docker configuration. Of course, you can also attach Docker-in-Podman to ChristopherHX/gitea-actions-runner to provide users with multiple choices.

ChristopherHX commented 8 months ago

Please create a PR to add your sample compose in examples/docker-compose-pind or examples/docker-compose-pinp. Maybe skip adding (or comment it out with a description) ./containers/registries.conf:/etc/containers/registries.conf as this seems to be special configuration for your registry mirrors.

Yes your configuration of rootful podman in docker works for me too without mounting registries.conf. You are writing about Podman in Docker and Podman in Podman.

See here https://www.redhat.com/sysadmin/podman-inside-container.

oilrich25 commented 8 months ago

Please create a PR to add your sample compose in examples/docker-compose-pind or examples/docker-compose-pinp. Maybe skip adding (or comment it out with a description) ./containers/registries.conf:/etc/containers/registries.conf as this seems to be special configuration for your registry mirrors.

Yes your configuration of rootful podman in docker works for me too without mounting registries.conf. You are writing about Podman in Docker and Podman in Podman.

See here https://www.redhat.com/sysadmin/podman-inside-container.

I found that Podman in Docker still has bugs.

##[group]Run docker/setup-buildx-action@main
with:
  driver: docker-container
  buildkitd-flags: --allow-insecure-entitlement security.insecure --allow-insecure-entitlement network.host
  install: false
  use: true
  cleanup: true
env:
  CACHE_EPOCH: 1
  CCACHE_MAXFILES: 0
  CCACHE_MAXSIZE: 200M
  SCCACHE_CACHE_SIZE: 200M
##[endgroup]
##[group]Docker info
[command]/usr/bin/docker version
Client:
 Version:           24.0.6
 API version:       1.41 (downgraded from 1.43)
 Go version:        go1.20.7
 Git commit:        ed223bc
 Built:             Mon Sep  4 12:30:51 2023
 OS/Arch:           linux/amd64
 Context:           default
Server: linux/amd64/fedora-39
 Podman Engine:
  Version:          4.8.1
  APIVersion:       4.8.1
  Arch:             amd64
  BuildTime:        2023-12-05T12:00:50Z
  Experimental:     false
  GitCommit:        
  GoVersion:        go1.21.4
  KernelVersion:    4.14.294-220.533.amzn2.x86_64
  MinAPIVersion:    4.0.0
  Os:               linux
 Conmon:
  Version:          conmon version 2.1.8, commit: 
  Package:          conmon-2.1.8-2.fc39.x86_64
 OCI Runtime (crun):
  Version:          crun version 1.12
commit: ce429cb2e277d001c2179df1ac66a470f00802ae
rundir: /run/crun
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  Package:          crun-1.12-1.fc39.x86_64
 Engine:
  Version:          4.8.1
  API version:      1.41 (minimum version 1.24)
  Go version:       go1.21.4
  Git commit:       
  Built:            Tue Dec  5 12:00:50 2023
  OS/Arch:          linux/amd64
  Experimental:     false
[command]/usr/bin/docker info
Client:
 Version:    24.0.6
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2
    Path:     /usr/local/lib/docker/cli-plugins/docker-buildx
Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 1
 Server Version: 4.8.1
 Storage Driver: overlay
  Native Overlay Diff: false
  Using metacopy: false
  Supports shifting: true
  Supports volatile: true
  Backing Filesystem: xfs
  Supports d_type: true
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge macvlan ipvlan
  Log: k8s-file none passthrough journald
 Swarm: inactive
 Runtimes: kata krun ocijail runc runsc crun runj youki crun-wasm
 Default Runtime: crun
 Init Binary: 
 containerd version: 
 runc version: 
 init version: 
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 4.14.294-220.533.amzn2.x86_64
 Operating System: fedora
 OSType: linux
 Architecture: amd64
 CPUs: 20
 Total Memory: 23.54GiB
 Name: d8ed75b8f712
 ID: 34826688-dcb3-43be-8c8a-b8e768db5d4c
 Docker Root Dir: /var/lib/containers/storage
 Debug Mode: true
  File Descriptors: 15
  Goroutines: 11
  System Time: 2023-12-15T21:00:24.037578576Z
  EventsListeners: 0
 Experimental: true
 Insecure Registries:
  k8s.gcr.io
  quay.io
  docker.io
  gcr.io
  ghcr.io
 Live Restore Enabled: false
 Product License: Apache-2.0
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
##[endgroup]
##[group]Buildx version
[command]/usr/bin/docker buildx version
github.com/docker/buildx v0.11.2 9872040b6626fb7d87ef7296fd5b832e8cc2ad17
##[endgroup]
##[group]Creating a new builder instance
[command]/usr/bin/docker buildx create --name builder-d5b977a3-2ada-437a-ae1d-fa2ab09b6a59 --driver docker-container --buildkitd-flags --allow-insecure-entitlement security.insecure --allow-insecure-entitlement network.host --use
builder-d5b977a3-2ada-437a-ae1d-fa2ab09b6a59
##[endgroup]
##[group]Booting builder
[command]/usr/bin/docker buildx inspect --bootstrap --builder builder-d5b977a3-2ada-437a-ae1d-fa2ab09b6a59
#1 [internal] booting buildkit
#1 pulling image moby/buildkit:buildx-stable-1
#1 pulling image moby/buildkit:buildx-stable-1 2.3s done
#1 creating container buildx_buildkit_builder-d5b977a3-2ada-437a-ae1d-fa2ab09b6a590 0.1s done
#1 ERROR: Error response from daemon: container create: cannot set cgroup parent if not creating cgroups: invalid argument
------
 > [internal] booting buildkit:
------
Name:          builder-d5b977a3-2ada-437a-ae1d-fa2ab09b6a59
Driver:        docker-container
Last Activity: 2023-12-15 21:00:24 +0000 UTC
Nodes:
Name:      builder-d5b977a3-2ada-437a-ae1d-fa2ab09b6a590
Endpoint:  unix:///var/run/podman/podman.sock
Status:    inactive
Flags:     --allow-insecure-entitlement security.insecure --allow-insecure-entitlement network.host
Platforms: 
##[endgroup]
##[group]Inspect builder
{
  "nodes": [
    {
      "name": "builder-d5b977a3-2ada-437a-ae1d-fa2ab09b6a590",
      "endpoint": "unix:///var/run/podman/podman.sock",
      "status": "inactive",
      "buildkitd-flags": "--allow-insecure-entitlement security.insecure --allow-insecure-entitlement network.host"
    }
  ],
  "name": "builder-d5b977a3-2ada-437a-ae1d-fa2ab09b6a59",
  "driver": "docker-container",
  "lastActivity": "2023-12-15T21:00:24.000Z"
}
##[endgroup]
##[group]BuildKit version
##[warning]Error: Error: No such object: buildx_buildkit_builder-d5b977a3-2ada-437a-ae1d-fa2ab09b6a590
builder-d5b977a3-2ada-437a-ae1d-fa2ab09b6a590: undefined
##[endgroup]