search

rancher / k3os

Purpose-built OS for Kubernetes, fully managed by Kubernetes.
https://k3os.io
Apache License 2.0
3.5k stars 401 forks source link

Configure Private Registry without TLS #847

Closed jslay88 closed 2 years ago

jslay88 commented 2 years ago

Trying to get k3os running in my home lab, for which I have an insecure docker registry. I know the registry works, as it has existed for some time now, and I am pulling images from it with other Docker hosts in the network, configured to use it as an insecure registry. Following rancher/k3os#396 , I have been trying to get this working on cloud-init.yaml, on fresh instances (not existing k3os instances).

However, with k3os, I have had no such luck coming from a cloud-init.yaml

cloud-init.yaml

ssh_authorized_keys:
- github:jslay88
write_files:
- path: /var/lib/connman/default.config
  content: |-
    [service_eth0]
    Type=ethernet
    IPv4=10.3.2.11/255.255.252.0/10.3.0.1
    IPv6=off
    Nameservers=10.3.0.3
- content: |-
    mirrors:
      docker.home.jslay.net:
        endpoint:
          - "http://docker.home.jslay.net:5000"
  encoding: ""
  owner: ""
  path: /var/lib/rancher/k3s/registries.yaml
  permissions: ""
run_cmd:
- sudo mkdir /longhorn
- sudo mount -t ext4 /dev/vdb1 /longhorn
hostname: k3s-worker01
k3os:
  modules:
  - kvm
  - nvme
  sysctl:
    kernel.printk: "4 4 1 7"
    kernel.kptr_restrict: "1"
  dns_nameservers:
  - 10.3.0.3
  ntp_servers:
  - 0.us.pool.ntp.org
  - 1.us.pool.ntp.org
  password: apassword
  server_url: https://10.3.2.1:6443
  token: mysupersecrettoken
  k3s_args:
  - agent
  - "--private-registry=/var/lib/rancher/k3s/registries.yaml"

When trying to pull an image from my private registry, it tries to pull using HTTPS and I receive the following error

  Normal   Pulling    7s (x2 over 21s)  kubelet            Pulling image "docker.home.jslay.net:5000/jslay/test_img:1"
  Warning  Failed     7s (x2 over 21s)  kubelet            Failed to pull image "docker.home.jslay.net:5000/jslay/test_img:1": rpc error: code = Unknown desc = failed to pull and unpack image "docker.home.jslay.net:5000/jslay/test_img:1": failed to resolve reference "docker.home.jslay.net:5000/jslay/test_img:1": failed to do request: Head "https://docker.home.jslay.net:5000/v2/jslay/test_img/manifests/1": http: server gave HTTP response to HTTPS client
  Warning  Failed     7s (x2 over 21s)  kubelet            Error: ErrImagePull
jslay88 commented 2 years ago

The correct configuration is

mirrors:
  "docker.home.jslay.net:5000":
    endpoint:
      - "http://docker.home.jslay.net:5000"

Where as the documentation is quite misleading and shows:

mirrors:
  mycustomreg.com:
    endpoint:
      - "https://mycustomreg.com:5000"