Open willzhang opened 1 year ago
neolit123
commented
1 year ago as mentioned on the k/release ticket it's a complicated change and needs a KEP (proposal doc/plan). not clear if we want to do it. we could survey kubeadm users about it.
it will beak kubeadm users and needs a smooth transition period.
willzhang
commented
1 year ago can change to this imageRepository: registry.k8s.io/kubernetes
root@node1:~# kubeadm config print init-defaults
apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
token: abcdef.0123456789abcdef
ttl: 24h0m0s
usages:
- signing
- authentication
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 1.2.3.4
bindPort: 6443
nodeRegistration:
criSocket: unix:///var/run/containerd/containerd.sock
imagePullPolicy: IfNotPresent
name: node
taints: null
---
apiServer:
timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta3
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns: {}
etcd:
local:
dataDir: /var/lib/etcd
imageRepository: registry.k8s.io/kubernetes
kind: ClusterConfiguration
kubernetesVersion: 1.25.0
networking:
dnsDomain: cluster.local
serviceSubnet: 10.96.0.0/12
scheduler: {}or
root@node1:~# kubeadm config images list
registry.k8s.io/kubernetes/kube-apiserver:v1.25.6
registry.k8s.io/kubernetes/kube-controller-manager:v1.25.6
registry.k8s.io/kubernetes/kube-scheduler:v1.25.6
registry.k8s.io/kubernetes/kube-proxy:v1.25.6
registry.k8s.io/kubernetes/pause:3.8
registry.k8s.io/kubernetes/etcd:3.5.6-0
registry.k8s.io/coredns/coredns:v1.9.3
javiroman
commented
1 year ago I have the same problem with 1.26.1 @willzhang where can I change the "imageRepository" value?
$ kubeadm config images list registry.k8s.io/kube-apiserver:v1.26.1 registry.k8s.io/kube-controller-manager:v1.26.1 registry.k8s.io/kube-scheduler:v1.26.1 registry.k8s.io/kube-proxy:v1.26.1 registry.k8s.io/pause:3.9 registry.k8s.io/etcd:3.5.6-0 registry.k8s.io/coredns/coredns:v1.9.3
willzhang
commented
1 year ago I have the same problem with 1.26.1 @willzhang where can I change the "imageRepository" value?
$ kubeadm config images list
registry.k8s.io/kube-apiserver:v1.26.1
registry.k8s.io/kube-controller-manager:v1.26.1
registry.k8s.io/kube-scheduler:v1.26.1
registry.k8s.io/kube-proxy:v1.26.1
registry.k8s.io/pause:3.9
registry.k8s.io/etcd:3.5.6-0
registry.k8s.io/coredns/coredns:v1.9.3
We need wait official support this feature.
SataQiu
commented
1 year ago Currently, you can specify the base image repository with the --image-repository flag (maybe a workaround?)
For example:
kubeadm config images list --image-repository registry.k8s.io/kuberneteskubeadm init --image-repository=registry.k8s.io/kubernetesyes ,but Many cluster installation tools like kubepspray encapsulate kubeadm, it's hard to change kubeadm init.
and not only kubeadm init, in offline environment need more operation like image pull image tag image push.
majorinche
commented
1 year ago Currently, you can specify the base image repository with the
--image-repositoryflag (maybe a workaround?) For example:kubeadm config images list --image-repository registry.k8s.io/kuberneteskubeadm init --image-repository=registry.k8s.io/kubernetes
but it doesn't work for pause, it is still registry.k8s.io, did not change as specified by --image-repository. failed to get sandbox image \"registry.k8s.io/pause:3.6\": failed to pull image \"registry.k8s.io/pause:3.6\"
pacoxu
commented
1 year ago if you are using containerd, pause image is configured in /etc/containerd/config.toml.
myysophia
commented
1 year ago I have the same problem with 1.22.10 where can I change the "imageRepository" value?
my question: kubeadm config images pull error, my image already exists locally, why does kubeadm images pull still pull it?
1.confirm the images exist
[root@k8s-master-node1 ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
registry.myrepo.com/kainstall/kube-apiserver v1.22.10 4f5d508856b7 11 months ago 128MB
registry.myrepo.com/kainstall/kube-scheduler v1.22.10 d1249c1cae8c 11 months ago 52.7MB
registry.myrepo.com/kainstall/kube-controller-manager v1.22.10 5454a57b8516 11 months ago 122MB
registry.myrepo.com/kainstall/kube-proxy v1.22.10 cb930b7f07c8 11 months ago 104MB
registry.myrepo.com/kainstall/etcd 3.5.0-0 004811815584 22 months ago 295MB
registry.myrepo.com/kainstall/coredns v1.8.4 8d147537fb7d 23 months ago 47.6MB
registry.myrepo.com/kainstall/pause 3.5 ed210e3e4a5b 2 years ago 683kB2.use kubeadmcfg.yaml list images
[root@k8s-master-node1 ~]# kubeadm config images list --config=/etc/kubernetes/kubeadmcfg.yaml
registry.myrepo.com/kainstall/kube-apiserver:v1.22.10
registry.myrepo.com/kainstall/kube-controller-manager:v1.22.10
registry.myrepo.com/kainstall/kube-scheduler:v1.22.10
registry.myrepo.com/kainstall/kube-proxy:v1.22.10
registry.myrepo.com/kainstall/pause:3.5
registry.myrepo.com/kainstall/etcd:3.5.0-0
registry.myrepo.com/kainstall/coredns:v1.8.43. kubeadm config images pull error, my image already exists locally, why does kubeadm images pull still pull it?
[root@k8s-master-node1 ~]# kubeadm config images pull --config=/etc/kubernetes/kubeadmcfg.yaml --v=6
I0505 21:33:57.869432 10018 initconfiguration.go:247] loading configuration from "/etc/kubernetes/kubeadmcfg.yaml"
I0505 21:33:57.876371 10018 interface.go:431] Looking for default routes with IPv4 addresses
I0505 21:33:57.876396 10018 interface.go:436] Default route transits interface "eth1"
I0505 21:33:57.876615 10018 interface.go:208] Interface eth1 is up
I0505 21:33:57.876693 10018 interface.go:256] Interface "eth1" has 1 addresses :[10.50.10.21/24].
I0505 21:33:57.876744 10018 interface.go:223] Checking addr 10.50.10.21/24.
I0505 21:33:57.876759 10018 interface.go:230] IP found 10.50.10.21
I0505 21:33:57.876772 10018 interface.go:262] Found valid IPv4 address 10.50.10.21 for interface "eth1".
I0505 21:33:57.876785 10018 interface.go:442] Found active IP 10.50.10.21
exit status 1
output: E0505 21:34:49.021343 10116 remote_image.go:238] "PullImage from image service failed" err="rpc error: code = Unknown desc = failed to pull and unpack image \"registry.myrepo.com/kainstall/kube-apiserver:v1.22.10\": failed to resolve reference \"registry.myrepo.com/kainstall/kube-apiserver:v1.22.10\": failed to do request: Head \"https://registry.myrepo.com/v2/kainstall/kube-apiserver/manifests/v1.22.10\": dial tcp: lookup registry.myrepo.com on 10.0.2.3:53: read udp 10.0.2.15:47034->10.0.2.3:53: i/o timeout" image="registry.myrepo.com/kainstall/kube-apiserver:v1.22.10"
time="2023-05-05T21:34:49+08:00" level=fatal msg="pulling image: rpc error: code = Unknown desc = failed to pull and unpack image \"registry.myrepo.com/kainstall/kube-apiserver:v1.22.10\": failed to resolve reference \"registry.myrepo.com/kainstall/kube-apiserver:v1.22.10\": failed to do request: Head \"https://registry.myrepo.com/v2/kainstall/kube-apiserver/manifests/v1.22.10\": dial tcp: lookup registry.myrepo.com on 10.0.2.3:53: read udp 10.0.2.15:47034->10.0.2.3:53: i/o timeout"4. According to this judgment rule, it should not be pulled again. I'm not sure if the code I checked is correct:)
// PullImage will pull an image if it is not present locally
// retrying up to retries times
// it returns true if it attempted to pull, and any errors from pulling
func PullImage(image string, retries int) (bool, error) {
// once we have configurable log levels
// if this did not return an error, then the image exists locally
if err := exec.NewHostCmd("docker", "inspect", "--type=image", image); err == nil {
return false, nil
}
// otherwise try to pull it
var err error
if err = exec.NewHostCmd("docker", "pull", image).Run(); err != nil {
for i := 0; i < retries; i++ {
time.Sleep(time.Second * time.Duration(i+1))
if err = exec.NewHostCmd("docker", "pull", image).Run(); err == nil {
break
}
}
}5、here is my InitConfiguration and config
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
nodeRegistration:
criSocket: unix:///run/containerd/containerd.sock
kubeletExtraArgs:
runtime-cgroups: /system.slice/containerd.service
pod-infra-container-image: registry.myrepo.com/kainstall/pause:3.5
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
kubernetesVersion: 1.22.10
controlPlaneEndpoint: apiserver.cluster.local:6443
networking:
dnsDomain: cluster.local
podSubnet: 10.244.0.0/16
serviceSubnet: 10.96.0.0/16
imageRepository: registry.myrepo.com/kainstall
apiServer:
certSANs:
- 127.0.0.1
- apiserver.cluster.local
- 10.50.10.21
extraArgs:
event-ttl: '720h'
service-node-port-range: '30000-50000'
audit-log-maxage: '20'
audit-log-maxbackup: '10'
audit-log-maxsize: '100'
audit-log-path: /var/log/kube-audit/audit.log
audit-policy-file: /etc/kubernetes/audit-policy.yaml
extraVolumes:
- name: audit-config
hostPath: /etc/kubernetes/audit-policy.yaml
mountPath: /etc/kubernetes/audit-policy.yaml
readOnly: true
pathType: File
- name: audit-log
hostPath: /var/log/kube-audit
mountPath: /var/log/kube-audit
pathType: DirectoryOrCreate
controllerManager:
extraArgs:
bind-address: 0.0.0.0
node-cidr-mask-size: '24'
node-monitor-grace-period: '20s'
pod-eviction-timeout: '2m'
terminated-pod-gc-threshold: '30'
cluster-signing-duration: 87600h
feature-gates: RotateKubeletServerCertificate=true
extraVolumes:
scheduler:
extraArgs:
bind-address: 0.0.0.0@myysophia your problem seems to be different.
I find that you are using criSocket: unix:///run/containerd/containerd.sock in the configuration but use docker images for checking local image. That would be different.
You can use check image status of containerd with nerdctl`crictl` with the correct socket.
myysophia
commented
1 year ago @pacoxu Thank you for reminding:)
k8s-triage-robot
commented
7 months ago The Kubernetes project currently lacks enough contributors to adequately respond to all issues.
This bot triages un-triaged issues according to the following rules:
lifecycle/stale is appliedlifecycle/stale was applied, lifecycle/rotten is appliedlifecycle/rotten was applied, the issue is closedYou can:
/remove-lifecycle stale/closePlease send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale
k8s-triage-robot
commented
6 months ago The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.
This bot triages un-triaged issues according to the following rules:
lifecycle/stale is appliedlifecycle/stale was applied, lifecycle/rotten is appliedlifecycle/rotten was applied, the issue is closedYou can:
/remove-lifecycle rotten/closePlease send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle rotten
neolit123
commented
6 months ago adding frozen label. there is no plan for this and it's not an easy change.
if someone wants to write a KEP please go ahead. https://github.com/kubernetes/enhancements/blob/master/keps/README.md
Is this a BUG REPORT or FEATURE REQUEST?
FEATURE REQUEST
Versions
kubeadm version (use
kubeadm version): v1.26.0 Environment:kubectl version): v1.26.0uname -a): 5.xWhat happened?
as https://github.com/kubernetes/sig-release/issues/2146 discuss， the long image url
registry.k8s.io/kubernetes/kube-apiserver:v1.26.0have been ready, kubeadm init should use this url as default to pull images.What you expected to happen?
kubeadmin init cluster pull images from
registry.k8s.io/kubernetes/kube-xxas default action.How to reproduce it (as minimally and precisely as possible)?
here have one image list for kubernetes cluster installtion generate by kubespray offline scripts.
the problems i face:
1.I can not just replace
registry.k8s.ioquay.iodocker.ioto192.168.72.10 (local offline harbor registry), harbor not support shot image url withregistry.k8s.io/kube-apiserver:v1.25.5${kube_image_repo}to${kube_image_repo}/kubernetesbasicly, becauese some application use long format url, egregistry.k8s.io/sig-storage/csi-resizer:v1.3.0registry.k8s.io/kubernetes/kube-apiserver:v1.26.0, but as default kubeadm only pull fromregistry.k8s.io/kube-apiserver:v1.26.0, i must change kubeadm config again.so this deferente image url Leading to many dilemmas
Anything else we need to know?
1、This will solve the confusion of the image format and make the image of the core components of kuberenrtes consistent with the image of all other applications.
2、It is easier to move images in different places through variables and scripts without special configuration.