kubernetes-sigs / sig-windows-dev-tools

This is a batteries included local development environment for Kubernetes on Windows.
Apache License 2.0
80 stars 46 forks source link

docs: Update notes on Windows with WSL with important requirements #245

Closed mloskot closed 1 year ago

mloskot commented 1 year ago

Run the cluster from Windows host filesystem. Running it from WSL filesystem is likely to fail.


The current documentation is missing an important requirement for those who want to run the two-node cluster on Windows in WSL environment.

It is important to run make all inside sis-windows-dev-tools repo cloned on Windows filesystem and not WSL filesystem, and here it is why:

Run in WSL from WSL filesystem: Fail

Go to WSL terminal, then run the following sequence:

$ cd ~
$ pwd
/home/mloskot
$ git clone https://github.com/kubernetes-sigs/sig-windows-dev-tools.git
$ cd sig-windows-dev-tools/
$ make all
cni: calico
calico: 3.25.0; containerd: 1.6.15
==> winw1: VM not created. Moving on...
==> controlplane: VM not created. Moving on...
clean
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    24  100    24    0     0     32      0 --:--:-- --:--:-- --:--:--    32
Using Kubernetes version v1.26.4-3-2d4a3e29be572e from upstream
...snip...
vagrant phase
making mock kubejoin file to keep Vagrantfile happy in sync/shared
######################################
Retry vagrant up if the first time the windows node failed
Starting the control plane
######################################
cni: calico
calico: 3.25.0; containerd: 1.6.15
Bringing machine 'controlplane' up with 'virtualbox' provider...
==> controlplane: Box 'roboxes/ubuntu2004' could not be found. Attempting to find and install...
    controlplane: Box Provider: virtualbox
    controlplane: Box Version: >= 0
==> controlplane: Loading metadata for box 'roboxes/ubuntu2004'
    controlplane: URL: https://vagrantcloud.com/roboxes/ubuntu2004
==> controlplane: Adding box 'roboxes/ubuntu2004' (v4.2.14) for provider: virtualbox
    controlplane: Downloading: https://vagrantcloud.com/roboxes/boxes/ubuntu2004/versions/4.2.14/providers/virtualbox.box
    controlplane:
    controlplane: Calculating and comparing box checksum...
==> controlplane: Successfully added box 'roboxes/ubuntu2004' (v4.2.14) for 'virtualbox'!
There are errors in the configuration of this machine. Please fix
the following errors and try again:

vm:
* The host path of the shared folder is not supported from WSL. Host
path of the shared folder must be located on a file system with
DrvFs type. Host path: ./sync/shared
* The host path of the shared folder is not supported from WSL. Host
path of the shared folder must be located on a file system with
DrvFs type. Host path: ./forked
* The host path of the shared folder is not supported from WSL. Host
path of the shared folder must be located on a file system with
DrvFs type. Host path: ./sync/linux

make: *** [Makefile:47: 2-vagrant-up] Error 1

Run in WSL from Windows host filesystem: Success

Go to WSL terminal, then run the following sequence - notice F: drive location, not WSL $HOME:

$ cd /mnt/f/Kubernetes/
$ git clone https://github.com/kubernetes-sigs/sig-windows-dev-tools.git
$ cd sig-windows-dev-tools/
$ make all
cni: calico
calico: 3.25.0; containerd: 1.6.15
==> winw1: VM not created. Moving on...
==> controlplane: VM not created. Moving on...
clean
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    24  100    24    0     0     33      0 --:--:-- --:--:-- --:--:--    33
Using Kubernetes version v1.26.4-3-2d4a3e29be572e from upstream
...snip...
vagrant phase
making mock kubejoin file to keep Vagrantfile happy in sync/shared
######################################
Retry vagrant up if the first time the windows node failed
Starting the control plane
######################################
cni: calico
calico: 3.25.0; containerd: 1.6.15
Bringing machine 'controlplane' up with 'virtualbox' provider...
==> controlplane: Importing base box 'roboxes/ubuntu2004'...
==> controlplane: Matching MAC address for NAT networking...
==> controlplane: Checking if box 'roboxes/ubuntu2004' version '4.2.14' is up to date...
==> controlplane: Setting the name of the VM: sig-windows-dev-tools_controlplane_1681496915596_58063
==> controlplane: Clearing any previously set network interfaces...
==> controlplane: Preparing network interfaces based on configuration...
    controlplane: Adapter 1: nat
    controlplane: Adapter 2: hostonly
==> controlplane: Forwarding ports...
    controlplane: 22 (guest) => 2222 (host) (adapter 1)
==> controlplane: Running 'pre-boot' VM customizations...
==> controlplane: Booting VM...
==> controlplane: Waiting for machine to boot. This may take a few minutes...
    controlplane: SSH address: 127.0.0.1:2222
    controlplane: SSH username: vagrant
    controlplane: SSH auth method: private key
    controlplane: Warning: Connection reset. Retrying...
    controlplane: Warning: Connection aborted. Retrying...
    controlplane:
    controlplane: Vagrant insecure key detected. Vagrant will automatically replace
    controlplane: this with a newly generated keypair for better security.
    controlplane:
    controlplane: Inserting generated public key within guest...
    controlplane: Removing insecure key from the guest if it's present...
    controlplane: Key inserted! Disconnecting and reconnecting using new SSH key...
==> controlplane: Machine booted and ready!
[controlplane] A Virtualbox Guest Additions installation was found but no tools to rebuild or start them.
...snip...
    controlplane: Successfully set StrictAffinity to: true
    controlplane: NAME                                       READY   STATUS              RESTARTS   AGE
    controlplane: calico-kube-controllers-6b7b9c649d-vlnfs   0/1     Pending             0          34s
    controlplane: calico-node-7h64m                          0/1     Init:0/2            0          34s
    controlplane: calico-typha-6c6c55c864-7fc2s              0/1     ContainerCreating   0          34s
*********** vagrant up first run done ~~~~ ENTERING WINDOWS BRINGUP LOOP ***
cni: calico
calico: 3.25.0; containerd: 1.6.15
Bringing machine 'winw1' up with 'virtualbox' provider...
==> winw1: Box 'sig-windows-dev-tools/windows-2019' could not be found. Attempting to find and install...
    winw1: Box Provider: virtualbox
    winw1: Box Version: 1.0
==> winw1: Loading metadata for box 'sig-windows-dev-tools/windows-2019'
    winw1: URL: https://vagrantcloud.com/sig-windows-dev-tools/windows-2019
==> winw1: Adding box 'sig-windows-dev-tools/windows-2019' (v1.0) for provider: virtualbox
    winw1: Downloading: https://vagrantcloud.com/sig-windows-dev-tools/boxes/windows-2019/versions/1.0/providers/virtualbox.box
    winw1:
    winw1: Calculating and comparing box checksum...
==> winw1: Successfully added box 'sig-windows-dev-tools/windows-2019' (v1.0) for 'virtualbox'!
==> winw1: Importing base box 'sig-windows-dev-tools/windows-2019'...
==> winw1: Matching MAC address for NAT networking...
==> winw1: Checking if box 'sig-windows-dev-tools/windows-2019' version '1.0' is up to date...
==> winw1: Setting the name of the VM: sig-windows-dev-tools_winw1_1681501575218_19249
==> winw1: Fixed port collision for 3389 => 3389. Now on port 2200.
==> winw1: Fixed port collision for 22 => 2222. Now on port 2201.
==> winw1: Clearing any previously set network interfaces...
==> winw1: Preparing network interfaces based on configuration...
    winw1: Adapter 1: nat
    winw1: Adapter 2: hostonly
==> winw1: Forwarding ports...
    winw1: 3389 (guest) => 2200 (host) (adapter 1)
    winw1: 5985 (guest) => 55985 (host) (adapter 1)
    winw1: 5986 (guest) => 55986 (host) (adapter 1)
    winw1: 22 (guest) => 2201 (host) (adapter 1)
==> winw1: Running 'pre-boot' VM customizations...
==> winw1: Booting VM...
==> winw1: Waiting for machine to boot. This may take a few minutes...
    winw1: WinRM address: 127.0.0.1:55985
    winw1: WinRM username: vagrant
    winw1: WinRM execution_time_limit: PT2H
    winw1: WinRM transport: negotiate
Timed out while waiting for the machine to boot. This means that
Vagrant was unable to communicate with the guest machine within
the configured ("config.vm.boot_timeout" value) time period.

If you look above, you should be able to see the error(s) that
Vagrant had when attempting to connect to the machine. These errors
are usually good hints as to what may be wrong.

If you're using a custom box, make sure that networking is properly
working and you're able to connect to the machine. It is a common
problem that networking isn't setup properly in these boxes.
Verify that authentication configurations are also setup properly,
as well.

If the box appears to be booting properly, you may want to increase
the timeout ("config.vm.boot_timeout") value.
failed_win_up
cni: calico
calico: 3.25.0; containerd: 1.6.15
==> winw1: Running provisioner: shell...
    winw1: Running: sync/windows/0-containerd.ps1 as C:\tmp\vagrant-shell.ps1
    winw1: Stopping  ContainerD & Kubelet
    winw1: Downloading Calico using ContainerD - [calico: 3.25] [containerd: 1.6.15]
    winw1: Installing 7Zip
    winw1: Getting ContainerD binaries
    winw1: Downloading https://github.com/containerd/containerd/releases/download/v1.6.15/containerd-1.6.15-windows-amd64.tar.gz to C:\Program Files\containerd\containerd.tar.gz
    winw1: x containerd-shim-runhcs-v1.exe
    winw1: x ctr.exe
    winw1: x containerd-stress.exe
    winw1: x containerd.exe: Can't unlink already-existing object
    winw1: tar.exe: Error exit delayed from previous errors.
    winw1: Registering ContainerD as a service
    winw1: time="2023-04-14T13:04:25.116345600-07:00" level=fatal msg="The specified service already exists."
    winw1: Starting ContainerD service
    winw1: Done - please remember to add '--cri-socket "npipe:////./pipe/containerd-containerd"' to your kubeadm join command
==> winw1: Running provisioner: shell...
    winw1: Running: sync/windows/forked.ps1 as C:\tmp\vagrant-shell.ps1
    winw1:
    winw1:
    winw1:     Directory: C:\
    winw1:
    winw1:
    winw1: Mode                LastWriteTime         Length Name
    winw1: ----                -------------         ------ ----
    winw1: d-----        1/21/2022   3:44 AM                k
    winw1:
    winw1:
    winw1: cp : Cannot find path 'C:\forked\StartKubelet.ps1' because it does not exist.
    winw1: At C:\tmp\vagrant-shell.ps1:11 char:5
    winw1: +     cp C:/forked/StartKubelet.ps1 c:\k\StartKubelet.ps1
    winw1: +     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    winw1:     + CategoryInfo          : ObjectNotFound: (C:\forked\StartKubelet.ps1:String) [Copy-Item], ItemNotFoundException
    winw1:     + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.CopyItemCommand
    winw1:
==> winw1: Running provisioner: shell...
    winw1: Running: sync/shared/kubejoin.ps1 as C:\tmp\vagrant-shell.ps1
    winw1: cp : Cannot find path 'C:\sync\windows\bin' because it does not exist.
    winw1: At C:\tmp\vagrant-shell.ps1:2 char:1
    winw1: + cp C:\sync\windows\bin\* c:\k
    winw1: + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    winw1:     + CategoryInfo          : ObjectNotFound: (C:\sync\windows\bin:String) [Copy-Item], ItemNotFoundException
    winw1:     + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.CopyItemCommand
    winw1:
    winw1: [preflight] Running pre-flight checks
    winw1: [preflight] WARNING: Couldn't create the interface used for talking to the container runtime: crictl is required for container runtime: exec: "crictl": executable file not found in %PATH%
    winw1: error execution phase preflight: [preflight] Some fatal errors occurred:
    winw1:      [ERROR FileAvailable-\etc\kubernetes\kubelet.conf]: \etc\kubernetes\kubelet.conf already exists
    winw1:      [ERROR FileAvailable-C:-etc-kubernetes-pki-ca.crt]: C:/etc/kubernetes/pki/ca.crt already exists
    winw1: [preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
    winw1: To see the stack trace of this error execute with --v=5 or higher
...snip...
mloskot commented 1 year ago

@aravindhp Happy to help, if I can. The make all is still running for me, so the section title Run in WSL from Windows host filesystem: Success above is not entirely true as it's not completed with success yet i.e. two-node cluster running. I'm updating the output with some Cannot find path failures as you can already see yourself. However, those failures seem not related to the actual issue this PR addresses.


I'm seeing the make all procedure being stuck in loop trying to resolve this issue - it's repeated the same step at least three times already - but as mentioned, this must be a different issue

...
cni: calico
calico: 3.25.0; containerd: 1.6.15
==> winw1: Running provisioner: shell...
    winw1: Running: sync/windows/0-containerd.ps1 as C:\tmp\vagrant-shell.ps1
    winw1: Stopping  ContainerD & Kubelet
    winw1: Downloading Calico using ContainerD - [calico: 3.25] [containerd: 1.6.15]
    winw1: Installing 7Zip
    winw1: Getting ContainerD binaries
    winw1: Downloading https://github.com/containerd/containerd/releases/download/v1.6.15/containerd-1.6.15-windows-amd64.tar.gz to C:\Program Files\containerd\containerd.tar.gz
    winw1: x containerd-shim-runhcs-v1.exe
    winw1: x ctr.exe
    winw1: x containerd-stress.exe
    winw1: x containerd.exe
    winw1: Registering ContainerD as a service
    winw1: Starting ContainerD service
    winw1: time="2023-04-14T13:23:12.295397500-07:00" level=fatal msg="The specified service already exists."
    winw1: Done - please remember to add '--cri-socket "npipe:////./pipe/containerd-containerd"' to your kubeadm join command
==> winw1: Running provisioner: shell...
    winw1: Running: sync/windows/forked.ps1 as C:\tmp\vagrant-shell.ps1
    winw1:
    winw1:
    winw1:     Directory: C:\
    winw1:
    winw1:
    winw1: Mode                LastWriteTime         Length Name
    winw1: ----                -------------         ------ ----
    winw1: d-----        1/21/2022   3:44 AM                k
    winw1:
    winw1:
    winw1: cp : Cannot find path 'C:\forked\StartKubelet.ps1' because it does not exist.
    winw1: At C:\tmp\vagrant-shell.ps1:11 char:5
    winw1: +     cp C:/forked/StartKubelet.ps1 c:\k\StartKubelet.ps1
    winw1: +     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    winw1:     + CategoryInfo          : ObjectNotFound: (C:\forked\StartKubelet.ps1:String) [Copy-Item], ItemNotFoundException
    winw1:     + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.CopyItemCommand
    winw1:
==> winw1: Running provisioner: shell...
...
aravindhp commented 1 year ago

/hold /lgtm cancel

OK, lets figure out the issue before we get this merged.

mloskot commented 1 year ago

Update: Success 1 (workstation DEV848)

@aravindhp After a few attempts and some tweaks, I've managed to run the cluster with success, I think. Next, I am going to try reproduce this success on a different Windows 11 machine.

Below, I copy full story in details from my personal notes at https://github.com/mloskot/sig-windows-dev-tools/wiki/Successful-Run-1

0. Host Environment

1. Patch variables.yaml

NOTICE: I don't know if these changes have been helpful or essential for the successful run, but after initial failures with the Windows node (see above), I went for a blind shot from the hip and bumped the versions.

Diff with variables.yaml changes ```diff diff --git a/variables.yaml b/variables.yaml index f1ffd68..1fbdff2 100644 --- a/variables.yaml +++ b/variables.yaml @@ -1,5 +1,5 @@ # Kubernetes version, pick major and minor, patch will be the latest released. -kubernetes_version: "1.26" +kubernetes_version: "1.27" # if this options is used BINARIES will be downloaded instead of building from zero build_from_source: "false" @@ -8,16 +8,16 @@ build_from_source: "false" pod_cidr: "100.244.0.0/16" # containerd version -containerd_version: "1.6.15" -calico_version: "3.25.0" +containerd_version: "1.7.0" +calico_version: "3.25.1" ## Linux settings k8s_linux_kubelet_nodeip: "10.20.30.10" windows_node_ip: "10.20.30.11" -windows_ram: 6048 -linux_ram: 4096 -linux_cpus: 2 -windows_cpus: 4 +windows_ram: 8192 +linux_ram: 8192 +linux_cpus: 4 +windows_cpus: 8 ## CHOOSE YOUR CNI ($) # cni: "calico" || "antrea" ```

2. Run make all

Inside WSL terminal (Ubuntu 22.04) run:

export VAGRANT=/mnt/c/HashiCorp/Vagrant/bin/vagrant.exe
cd sig-windows-dev-tools
make all
Full make all log ``` $ make all cni: calico ==> winw1: VM not created. Moving on... ==> controlplane: VM not created. Moving on... clean % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 24 100 24 0 0 31 0 --:--:-- --:--:-- --:--:-- 31 Using Kubernetes version v1.27.1-1-95feac5269be09 from upstream build ...snip... vagrant phase making mock kubejoin file to keep Vagrantfile happy in sync/shared ###################################### Retry vagrant up if the first time the windows node failed Starting the control plane ###################################### cni: calico calico: 3.25.1; containerd: 1.7.0 Bringing machine 'controlplane' up with 'virtualbox' provider... ==> controlplane: Importing base box 'roboxes/ubuntu2004'... ==> controlplane: Matching MAC address for NAT networking... ==> controlplane: Checking if box 'roboxes/ubuntu2004' version '4.2.14' is up to date... ==> controlplane: Setting the name of the VM: sig-windows-dev-tools_controlplane_1681513091473_37066 ==> controlplane: Clearing any previously set network interfaces... ==> controlplane: Preparing network interfaces based on configuration... controlplane: Adapter 1: nat controlplane: Adapter 2: hostonly ==> controlplane: Forwarding ports... controlplane: 22 (guest) => 2222 (host) (adapter 1) ==> controlplane: Running 'pre-boot' VM customizations... ==> controlplane: Booting VM... ==> controlplane: Waiting for machine to boot. This may take a few minutes... controlplane: SSH address: 127.0.0.1:2222 controlplane: SSH username: vagrant controlplane: SSH auth method: private key controlplane: controlplane: Vagrant insecure key detected. Vagrant will automatically replace controlplane: this with a newly generated keypair for better security. controlplane: controlplane: Inserting generated public key within guest... controlplane: Removing insecure key from the guest if it's present... controlplane: Key inserted! Disconnecting and reconnecting using new SSH key... ==> controlplane: Machine booted and ready! [controlplane] A Virtualbox Guest Additions installation was found but no tools to rebuild or start them. Reading package lists... ...snip... update-initramfs: Generating /boot/initrd.img-5.4.0-139-generic Unmounting Virtualbox Guest Additions ISO from: /mnt ==> controlplane: Checking for guest additions in VM... ==> controlplane: Setting hostname... ==> controlplane: Configuring and enabling network interfaces... ==> controlplane: Mounting shared folders... controlplane: /var/sync/linux => F:/Kubernetes/sig-windows-dev-tools/sync/linux controlplane: /var/sync/forked => F:/Kubernetes/sig-windows-dev-tools/forked controlplane: /var/sync/shared => F:/Kubernetes/sig-windows-dev-tools/sync/shared ==> controlplane: Running provisioner: shell... controlplane: Running: C:/Users/mateuszl/AppData/Local/Temp/vagrant-shell20230415-22620-zki8o8.sh controlplane: ARGS: 1.27 10.20.30.10 100.244.0.0/16 controlplane: Using 1.27 as the Kubernetes version controlplane: Setting up internet connectivity to /etc/resolv.conf controlplane: nameserver 8.8.8.8 controlplane: nameserver 1.1.1.1 controlplane: now curling to add keys... controlplane: Warning: apt-key output should not be parsed (stdout is not a terminal) controlplane: OK controlplane: deb https://apt.kubernetes.io/ kubernetes-xenial main controlplane: SWDT: Running apt get update -y controlplane: Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease controlplane: Hit:2 http://us.archive.ubuntu.com/ubuntu focal InRelease controlplane: Hit:4 http://us.archive.ubuntu.com/ubuntu focal-updates InRelease controlplane: Hit:5 http://us.archive.ubuntu.com/ubuntu focal-backports InRelease controlplane: Get:3 https://packages.cloud.google.com/apt kubernetes-xenial InRelease [8,993 B] controlplane: Get:6 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 Packages [65.7 kB] controlplane: Fetched 74.7 kB in 3s (21.7 kB/s) controlplane: Reading package lists... controlplane: overlay controlplane: br_netfilter controlplane: SWDT: Running modprobes controlplane: net.bridge.bridge-nf-call-iptables = 1 controlplane: net.ipv4.ip_forward = 1 controlplane: net.bridge.bridge-nf-call-ip6tables = 1 controlplane: * Applying /etc/sysctl.d/10-console-messages.conf ... controlplane: kernel.printk = 4 4 1 7 controlplane: * Applying /etc/sysctl.d/10-ipv6-privacy.conf ... controlplane: net.ipv6.conf.all.use_tempaddr = 2 controlplane: net.ipv6.conf.default.use_tempaddr = 2 controlplane: * Applying /etc/sysctl.d/10-kernel-hardening.conf ... controlplane: kernel.kptr_restrict = 1 controlplane: * Applying /etc/sysctl.d/10-link-restrictions.conf ... controlplane: fs.protected_hardlinks = 1 controlplane: fs.protected_symlinks = 1 controlplane: * Applying /etc/sysctl.d/10-magic-sysrq.conf ... controlplane: kernel.sysrq = 176 controlplane: * Applying /etc/sysctl.d/10-network-security.conf ... controlplane: net.ipv4.conf.default.rp_filter = 2 controlplane: net.ipv4.conf.all.rp_filter = 2 controlplane: * Applying /etc/sysctl.d/10-ptrace.conf ... controlplane: kernel.yama.ptrace_scope = 1 controlplane: * Applying /etc/sysctl.d/10-zeropage.conf ... controlplane: vm.mmap_min_addr = 65536 controlplane: * Applying /usr/lib/sysctl.d/50-default.conf ... controlplane: net.ipv4.conf.default.promote_secondaries = 1 controlplane: sysctl: setting key "net.ipv4.conf.all.promote_secondaries": Invalid argument controlplane: net.ipv4.ping_group_range = 0 2147483647 controlplane: net.core.default_qdisc = fq_codel controlplane: fs.protected_regular = 1 controlplane: fs.protected_fifos = 1 controlplane: * Applying /usr/lib/sysctl.d/50-pid-max.conf ... controlplane: kernel.pid_max = 4194304 controlplane: * Applying /etc/sysctl.d/99-kubernetes-cri.conf ... controlplane: net.bridge.bridge-nf-call-iptables = 1 controlplane: net.ipv4.ip_forward = 1 controlplane: net.bridge.bridge-nf-call-ip6tables = 1 controlplane: * Applying /etc/sysctl.d/99-sysctl.conf ... controlplane: net.ipv6.conf.all.disable_ipv6 = 1 controlplane: * Applying /usr/lib/sysctl.d/protect-links.conf ... controlplane: fs.protected_fifos = 1 controlplane: fs.protected_hardlinks = 1 controlplane: fs.protected_regular = 2 controlplane: fs.protected_symlinks = 1 controlplane: * Applying /etc/sysctl.conf ... controlplane: net.ipv6.conf.all.disable_ipv6 = 1 controlplane: SWDT installing kubelet, kubeadm, kubectl will overwrite them later as needeed... controlplane: Reading package lists... controlplane: Building dependency tree... controlplane: Reading state information... controlplane: The following additional packages will be installed: controlplane: conntrack cri-tools ebtables kubernetes-cni socat controlplane: Suggested packages: controlplane: nftables controlplane: The following NEW packages will be installed: controlplane: conntrack cri-tools ebtables kubeadm kubectl kubelet kubernetes-cni socat controlplane: 0 upgraded, 8 newly installed, 0 to remove and 70 not upgraded. controlplane: Need to get 85.9 MB of archives. controlplane: After this operation, 329 MB of additional disk space will be used. controlplane: Get:1 http://us.archive.ubuntu.com/ubuntu focal/main amd64 conntrack amd64 1:1.4.5-2 [30.3 kB] controlplane: Get:2 http://us.archive.ubuntu.com/ubuntu focal/main amd64 ebtables amd64 2.0.11-3build1 [80.3 kB] controlplane: Get:6 http://us.archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] controlplane: Get:3 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 cri-tools amd64 1.26.0-00 [18.9 MB] controlplane: Get:4 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 kubernetes-cni amd64 1.2.0-00 [27.6 MB] controlplane: Get:5 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 kubelet amd64 1.27.1-00 [18.7 MB] controlplane: Get:7 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 kubectl amd64 1.27.1-00 [10.2 MB] controlplane: Get:8 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 kubeadm amd64 1.27.1-00 [9,928 kB] controlplane: dpkg-preconfigure: unable to re-open stdin: No such file or directory controlplane: Fetched 85.9 MB in 2min 52s (499 kB/s) controlplane: Selecting previously unselected package conntrack. (Reading database ... 117499 files and directories currently installed.) controlplane: Preparing to unpack .../0-conntrack_1%3a1.4.5-2_amd64.deb ... controlplane: Unpacking conntrack (1:1.4.5-2) ... controlplane: Selecting previously unselected package cri-tools. controlplane: Preparing to unpack .../1-cri-tools_1.26.0-00_amd64.deb ... controlplane: Unpacking cri-tools (1.26.0-00) ... controlplane: Selecting previously unselected package ebtables. controlplane: Preparing to unpack .../2-ebtables_2.0.11-3build1_amd64.deb ... controlplane: Unpacking ebtables (2.0.11-3build1) ... controlplane: Selecting previously unselected package kubernetes-cni. controlplane: Preparing to unpack .../3-kubernetes-cni_1.2.0-00_amd64.deb ... controlplane: Unpacking kubernetes-cni (1.2.0-00) ... controlplane: Selecting previously unselected package socat. controlplane: Preparing to unpack .../4-socat_1.7.3.3-2_amd64.deb ... controlplane: Unpacking socat (1.7.3.3-2) ... controlplane: Selecting previously unselected package kubelet. controlplane: Preparing to unpack .../5-kubelet_1.27.1-00_amd64.deb ... controlplane: Unpacking kubelet (1.27.1-00) ... controlplane: Selecting previously unselected package kubectl. controlplane: Preparing to unpack .../6-kubectl_1.27.1-00_amd64.deb ... controlplane: Unpacking kubectl (1.27.1-00) ... controlplane: Selecting previously unselected package kubeadm. controlplane: Preparing to unpack .../7-kubeadm_1.27.1-00_amd64.deb ... controlplane: Unpacking kubeadm (1.27.1-00) ... controlplane: Setting up conntrack (1:1.4.5-2) ... controlplane: Setting up kubectl (1.27.1-00) ... controlplane: Setting up ebtables (2.0.11-3build1) ... controlplane: Setting up socat (1.7.3.3-2) ... controlplane: Setting up cri-tools (1.26.0-00) ... controlplane: Setting up kubernetes-cni (1.2.0-00) ... controlplane: Setting up kubelet (1.27.1-00) ... controlplane: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service → /lib/systemd/system/kubelet.service. controlplane: Setting up kubeadm (1.27.1-00) ... controlplane: Processing triggers for man-db (2.9.1-1) ... controlplane: kubelet set on hold. controlplane: kubeadm set on hold. controlplane: kubectl set on hold. controlplane: Configuring Containerd controlplane: Reading package lists... controlplane: Building dependency tree... controlplane: Reading state information... controlplane: lsb-release is already the newest version (11.1.0ubuntu2). controlplane: ca-certificates is already the newest version (20211016ubuntu0.20.04.1). controlplane: gnupg is already the newest version (2.2.19-3ubuntu2.2). controlplane: 0 upgraded, 0 newly installed, 0 to remove and 70 not upgraded. controlplane: Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease controlplane: Hit:2 http://us.archive.ubuntu.com/ubuntu focal InRelease controlplane: Get:3 https://download.docker.com/linux/ubuntu focal InRelease [57.7 kB] controlplane: Hit:5 http://us.archive.ubuntu.com/ubuntu focal-updates InRelease controlplane: Hit:6 http://us.archive.ubuntu.com/ubuntu focal-backports InRelease controlplane: Get:7 https://download.docker.com/linux/ubuntu focal/stable amd64 Packages [26.8 kB] controlplane: Hit:4 https://packages.cloud.google.com/apt kubernetes-xenial InRelease controlplane: Fetched 84.5 kB in 2s (47.3 kB/s) controlplane: Reading package lists... controlplane: Reading package lists... controlplane: Building dependency tree... controlplane: Reading state information... controlplane: The following NEW packages will be installed: controlplane: containerd.io controlplane: 0 upgraded, 1 newly installed, 0 to remove and 70 not upgraded. controlplane: Need to get 28.3 MB of archives. controlplane: After this operation, 116 MB of additional disk space will be used. controlplane: Get:1 https://download.docker.com/linux/ubuntu focal/stable amd64 containerd.io amd64 1.6.20-1 [28.3 MB] controlplane: dpkg-preconfigure: unable to re-open stdin: No such file or directory controlplane: Fetched 28.3 MB in 2min 38s (179 kB/s) controlplane: Selecting previously unselected package containerd.io. (Reading database ... 117593 files and directories currently installed.) controlplane: Preparing to unpack .../containerd.io_1.6.20-1_amd64.deb ... controlplane: Unpacking containerd.io (1.6.20-1) ... controlplane: Setting up containerd.io (1.6.20-1) ... controlplane: Created symlink /etc/systemd/system/multi-user.target.wants/containerd.service → /lib/systemd/system/containerd.service. controlplane: Processing triggers for man-db (2.9.1-1) ... controlplane: copying /var/sync/linux/bin/kubeadm to node path.. controlplane: copying /var/sync/linux/bin/kubectl to node path.. controlplane: copying /var/sync/linux/bin/kubelet to node path.. controlplane: disabled_plugins = [] controlplane: imports = [] controlplane: oom_score = 0 controlplane: plugin_dir = "" controlplane: required_plugins = [] controlplane: root = "/var/lib/containerd" controlplane: state = "/run/containerd" controlplane: temp = "" controlplane: version = 2 controlplane: controlplane: [cgroup] controlplane: path = "" controlplane: controlplane: [debug] controlplane: address = "" controlplane: format = "" controlplane: gid = 0 controlplane: level = "" controlplane: uid = 0 controlplane: controlplane: [grpc] controlplane: address = "/run/containerd/containerd.sock" controlplane: gid = 0 controlplane: max_recv_message_size = 16777216 controlplane: max_send_message_size = 16777216 controlplane: tcp_address = "" controlplane: tcp_tls_ca = "" controlplane: tcp_tls_cert = "" controlplane: tcp_tls_key = "" controlplane: uid = 0 controlplane: controlplane: [metrics] controlplane: address = "" controlplane: grpc_histogram = false controlplane: controlplane: [plugins] controlplane: controlplane: [plugins."io.containerd.gc.v1.scheduler"] controlplane: deletion_threshold = 0 controlplane: mutation_threshold = 100 controlplane: pause_threshold = 0.02 controlplane: schedule_delay = "0s" controlplane: startup_delay = "100ms" controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri"] controlplane: device_ownership_from_security_context = false controlplane: disable_apparmor = false controlplane: disable_cgroup = false controlplane: disable_hugetlb_controller = true controlplane: disable_proc_mount = false controlplane: disable_tcp_service = true controlplane: enable_selinux = false controlplane: enable_tls_streaming = false controlplane: enable_unprivileged_icmp = false controlplane: enable_unprivileged_ports = false controlplane: ignore_image_defined_volumes = false controlplane: max_concurrent_downloads = 3 controlplane: max_container_log_line_size = 16384 controlplane: netns_mounts_under_state_dir = false controlplane: restrict_oom_score_adj = false controlplane: sandbox_image = "registry.k8s.io/pause:3.6" controlplane: selinux_category_range = 1024 controlplane: stats_collect_period = 10 controlplane: stream_idle_timeout = "4h0m0s" controlplane: stream_server_address = "127.0.0.1" controlplane: stream_server_port = "0" controlplane: systemd_cgroup = false controlplane: tolerate_missing_hugetlb_controller = true controlplane: unset_seccomp_profile = "" controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".cni] controlplane: bin_dir = "/opt/cni/bin" controlplane: conf_dir = "/etc/cni/net.d" controlplane: conf_template = "" controlplane: ip_pref = "" controlplane: max_conf_num = 1 controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".containerd] controlplane: default_runtime_name = "runc" controlplane: disable_snapshot_annotations = true controlplane: discard_unpacked_layers = false controlplane: ignore_rdt_not_enabled_errors = false controlplane: no_pivot = false controlplane: snapshotter = "overlayfs" controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".containerd.default_runtime] controlplane: base_runtime_spec = "" controlplane: cni_conf_dir = "" controlplane: cni_max_conf_num = 0 controlplane: container_annotations = [] controlplane: pod_annotations = [] controlplane: privileged_without_host_devices = false controlplane: runtime_engine = "" controlplane: runtime_path = "" controlplane: runtime_root = "" controlplane: runtime_type = "" controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".containerd.default_runtime.options] controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".containerd.runtimes] controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc] controlplane: base_runtime_spec = "" controlplane: cni_conf_dir = "" controlplane: cni_max_conf_num = 0 controlplane: container_annotations = [] controlplane: pod_annotations = [] controlplane: privileged_without_host_devices = false controlplane: runtime_engine = "" controlplane: runtime_path = "" controlplane: runtime_root = "" controlplane: runtime_type = "io.containerd.runc.v2" controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options] controlplane: BinaryName = "" controlplane: CriuImagePath = "" controlplane: CriuPath = "" controlplane: CriuWorkPath = "" controlplane: IoGid = 0 controlplane: IoUid = 0 controlplane: NoNewKeyring = false controlplane: NoPivotRoot = false controlplane: Root = "" controlplane: ShimCgroup = "" controlplane: SystemdCgroup = false controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".containerd.untrusted_workload_runtime] controlplane: base_runtime_spec = "" controlplane: cni_conf_dir = "" controlplane: cni_max_conf_num = 0 controlplane: container_annotations = [] controlplane: pod_annotations = [] controlplane: privileged_without_host_devices = false controlplane: runtime_engine = "" controlplane: runtime_path = "" controlplane: runtime_root = "" controlplane: runtime_type = "" controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".containerd.untrusted_workload_runtime.options] controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".image_decryption] controlplane: key_model = "node" controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".registry] controlplane: config_path = "" controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".registry.auths] controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".registry.configs] controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".registry.headers] controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".registry.mirrors] controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".x509_key_pair_streaming] controlplane: tls_cert_file = "" controlplane: tls_key_file = "" controlplane: controlplane: [plugins."io.containerd.internal.v1.opt"] controlplane: path = "/opt/containerd" controlplane: controlplane: [plugins."io.containerd.internal.v1.restart"] controlplane: interval = "10s" controlplane: controlplane: [plugins."io.containerd.internal.v1.tracing"] controlplane: sampling_ratio = 1.0 controlplane: service_name = "containerd" controlplane: controlplane: [plugins."io.containerd.metadata.v1.bolt"] controlplane: content_sharing_policy = "shared" controlplane: controlplane: [plugins."io.containerd.monitor.v1.cgroups"] controlplane: no_prometheus = false controlplane: controlplane: [plugins."io.containerd.runtime.v1.linux"] controlplane: no_shim = false controlplane: runtime = "runc" controlplane: runtime_root = "" controlplane: shim = "containerd-shim" controlplane: shim_debug = false controlplane: controlplane: [plugins."io.containerd.runtime.v2.task"] controlplane: platforms = ["linux/amd64"] controlplane: sched_core = false controlplane: controlplane: [plugins."io.containerd.service.v1.diff-service"] controlplane: default = ["walking"] controlplane: controlplane: [plugins."io.containerd.service.v1.tasks-service"] controlplane: rdt_config_file = "" controlplane: controlplane: [plugins."io.containerd.snapshotter.v1.aufs"] controlplane: root_path = "" controlplane: controlplane: [plugins."io.containerd.snapshotter.v1.btrfs"] controlplane: root_path = "" controlplane: controlplane: [plugins."io.containerd.snapshotter.v1.devmapper"] controlplane: async_remove = false controlplane: base_image_size = "" controlplane: discard_blocks = false controlplane: fs_options = "" controlplane: fs_type = "" controlplane: pool_name = "" controlplane: root_path = "" controlplane: controlplane: [plugins."io.containerd.snapshotter.v1.native"] controlplane: root_path = "" controlplane: controlplane: [plugins."io.containerd.snapshotter.v1.overlayfs"] controlplane: root_path = "" controlplane: upperdir_label = false controlplane: controlplane: [plugins."io.containerd.snapshotter.v1.zfs"] controlplane: root_path = "" controlplane: controlplane: [plugins."io.containerd.tracing.processor.v1.otlp"] controlplane: endpoint = "" controlplane: insecure = false controlplane: protocol = "" controlplane: controlplane: [proxy_plugins] controlplane: controlplane: [stream_processors] controlplane: controlplane: [stream_processors."io.containerd.ocicrypt.decoder.v1.tar"] controlplane: accepts = ["application/vnd.oci.image.layer.v1.tar+encrypted"] controlplane: args = ["--decryption-keys-path", "/etc/containerd/ocicrypt/keys"] controlplane: env = ["OCICRYPT_KEYPROVIDER_CONFIG=/etc/containerd/ocicrypt/ocicrypt_keyprovider.conf"] controlplane: path = "ctd-decoder" controlplane: returns = "application/vnd.oci.image.layer.v1.tar" controlplane: controlplane: [stream_processors."io.containerd.ocicrypt.decoder.v1.tar.gzip"] controlplane: accepts = ["application/vnd.oci.image.layer.v1.tar+gzip+encrypted"] controlplane: args = ["--decryption-keys-path", "/etc/containerd/ocicrypt/keys"] controlplane: env = ["OCICRYPT_KEYPROVIDER_CONFIG=/etc/containerd/ocicrypt/ocicrypt_keyprovider.conf"] controlplane: path = "ctd-decoder" controlplane: returns = "application/vnd.oci.image.layer.v1.tar+gzip" controlplane: controlplane: [timeouts] controlplane: "io.containerd.timeout.bolt.open" = "0s" controlplane: "io.containerd.timeout.shim.cleanup" = "5s" controlplane: "io.containerd.timeout.shim.load" = "5s" controlplane: "io.containerd.timeout.shim.shutdown" = "3s" controlplane: "io.containerd.timeout.task.state" = "2s" controlplane: controlplane: [ttrpc] controlplane: address = "" controlplane: gid = 0 controlplane: uid = 0 controlplane: I0414 23:09:19.759551 39370 initconfiguration.go:255] loading configuration from "/var/sync/shared/kubeadm.yaml" controlplane: I0414 23:09:19.761847 39370 initconfiguration.go:117] detected and using CRI socket: unix:///var/run/containerd/containerd.sock controlplane: I0414 23:09:19.761926 39370 kubelet.go:196] the value of KubeletConfiguration.cgroupDriver is empty; setting it to "systemd" controlplane: I0414 23:09:19.767690 39370 version.go:187] fetching Kubernetes version from URL: https://dl.k8s.io/release/stable-1.27.txt controlplane: [init] Using Kubernetes version: v1.27.1 controlplane: [preflight] Running pre-flight checks controlplane: I0414 23:09:21.935291 39370 common.go:128] WARNING: tolerating control plane version v1.27.1 as a pre-release version controlplane: I0414 23:09:21.935706 39370 checks.go:563] validating Kubernetes and kubeadm version controlplane: I0414 23:09:21.935721 39370 checks.go:168] validating if the firewall is enabled and active controlplane: I0414 23:09:21.949381 39370 checks.go:203] validating availability of port 6443 controlplane: I0414 23:09:21.949675 39370 checks.go:203] validating availability of port 10259 controlplane: I0414 23:09:21.949766 39370 checks.go:203] validating availability of port 10257 controlplane: I0414 23:09:21.949783 39370 checks.go:280] validating the existence of file /etc/kubernetes/manifests/kube-apiserver.yaml controlplane: I0414 23:09:21.949800 39370 checks.go:280] validating the existence of file /etc/kubernetes/manifests/kube-controller-manager.yaml controlplane: I0414 23:09:21.949807 39370 checks.go:280] validating the existence of file /etc/kubernetes/manifests/kube-scheduler.yaml controlplane: I0414 23:09:21.949813 39370 checks.go:280] validating the existence of file /etc/kubernetes/manifests/etcd.yaml controlplane: I0414 23:09:21.949820 39370 checks.go:430] validating if the connectivity type is via proxy or direct controlplane: I0414 23:09:21.949835 39370 checks.go:469] validating http connectivity to first IP address in the CIDR controlplane: I0414 23:09:21.949851 39370 checks.go:469] validating http connectivity to first IP address in the CIDR controlplane: I0414 23:09:21.949858 39370 checks.go:104] validating the container runtime controlplane: I0414 23:09:22.007332 39370 checks.go:639] validating whether swap is enabled or not controlplane: I0414 23:09:22.007427 39370 checks.go:370] validating the presence of executable crictl controlplane: I0414 23:09:22.007449 39370 checks.go:370] validating the presence of executable conntrack controlplane: I0414 23:09:22.007470 39370 checks.go:370] validating the presence of executable ip controlplane: I0414 23:09:22.007484 39370 checks.go:370] validating the presence of executable iptables controlplane: I0414 23:09:22.007523 39370 checks.go:370] validating the presence of executable mount controlplane: I0414 23:09:22.007533 39370 checks.go:370] validating the presence of executable nsenter controlplane: I0414 23:09:22.007556 39370 checks.go:370] validating the presence of executable ebtables controlplane: I0414 23:09:22.007630 39370 checks.go:370] validating the presence of executable ethtool controlplane: I0414 23:09:22.007647 39370 checks.go:370] validating the presence of executable socat controlplane: I0414 23:09:22.007660 39370 checks.go:370] validating the presence of executable tc controlplane: I0414 23:09:22.007675 39370 checks.go:370] validating the presence of executable touch controlplane: I0414 23:09:22.007686 39370 checks.go:516] running all checks controlplane: I0414 23:09:22.028711 39370 checks.go:401] checking whether the given node name is valid and reachable using net.LookupHost controlplane: I0414 23:09:22.028777 39370 checks.go:605] validating kubelet version controlplane: I0414 23:09:22.183331 39370 checks.go:130] validating if the "kubelet" service is enabled and active controlplane: I0414 23:09:22.196626 39370 checks.go:203] validating availability of port 10250 controlplane: I0414 23:09:22.196760 39370 checks.go:329] validating the contents of file /proc/sys/net/bridge/bridge-nf-call-iptables controlplane: [preflight] Pulling images required for setting up a Kubernetes cluster controlplane: [preflight] This might take a minute or two, depending on the speed of your internet connection controlplane: [preflight] You can also perform this action in beforehand using 'kubeadm config images pull' controlplane: I0414 23:09:22.196865 39370 checks.go:329] validating the contents of file /proc/sys/net/ipv4/ip_forward controlplane: I0414 23:09:22.196885 39370 checks.go:203] validating availability of port 2379 controlplane: I0414 23:09:22.196907 39370 checks.go:203] validating availability of port 2380 controlplane: I0414 23:09:22.196921 39370 checks.go:243] validating the existence and emptiness of directory /var/lib/etcd controlplane: W0414 23:09:22.200379 39370 images.go:80] could not find officially supported version of etcd for Kubernetes v1.27.1, falling back to the nearest etcd version (3.5.7-0) controlplane: I0414 23:09:22.200460 39370 checks.go:828] using image pull policy: IfNotPresent controlplane: I0414 23:09:22.250946 39370 checks.go:854] pulling: registry.k8s.io/kube-apiserver:v1.27.1 controlplane: I0414 23:10:46.164268 39370 checks.go:854] pulling: registry.k8s.io/kube-controller-manager:v1.27.1 controlplane: I0414 23:11:48.151716 39370 checks.go:854] pulling: registry.k8s.io/kube-scheduler:v1.27.1 controlplane: I0414 23:12:21.827668 39370 checks.go:854] pulling: registry.k8s.io/kube-proxy:v1.27.1 controlplane: W0414 23:12:52.126987 39370 checks.go:835] detected that the sandbox image "registry.k8s.io/pause:3.6" of the container runtime is inconsistent with that used by kubeadm. It is recommended that using "registry.k8s.io/pause:3.9" as the CRI sandbox image. controlplane: I0414 23:12:52.193118 39370 checks.go:854] pulling: registry.k8s.io/pause:3.9 controlplane: I0414 23:12:58.684420 39370 checks.go:854] pulling: registry.k8s.io/etcd:3.5.7-0 controlplane: I0414 23:15:56.873636 39370 checks.go:854] pulling: registry.k8s.io/coredns/coredns:v1.10.1 controlplane: [certs] Using certificateDir folder "/etc/kubernetes/pki" controlplane: I0414 23:16:24.487980 39370 certs.go:112] creating a new certificate authority for ca controlplane: [certs] Generating "ca" certificate and key controlplane: I0414 23:16:24.717176 39370 certs.go:519] validating certificate period for ca certificate controlplane: [certs] Generating "apiserver" certificate and key controlplane: [certs] apiserver serving cert is signed for DNS names [controlplane kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 10.20.30.10] controlplane: [certs] Generating "apiserver-kubelet-client" certificate and key controlplane: I0414 23:16:25.105668 39370 certs.go:112] creating a new certificate authority for front-proxy-ca controlplane: [certs] Generating "front-proxy-ca" certificate and key controlplane: I0414 23:16:25.382977 39370 certs.go:519] validating certificate period for front-proxy-ca certificate controlplane: [certs] Generating "front-proxy-client" certificate and key controlplane: I0414 23:16:25.588490 39370 certs.go:112] creating a new certificate authority for etcd-ca controlplane: [certs] Generating "etcd/ca" certificate and key controlplane: I0414 23:16:25.970338 39370 certs.go:519] validating certificate period for etcd/ca certificate controlplane: [certs] Generating "etcd/server" certificate and key controlplane: [certs] etcd/server serving cert is signed for DNS names [controlplane localhost] and IPs [10.20.30.10 127.0.0.1 ::1] controlplane: [certs] Generating "etcd/peer" certificate and key controlplane: [certs] etcd/peer serving cert is signed for DNS names [controlplane localhost] and IPs [10.20.30.10 127.0.0.1 ::1] controlplane: [certs] Generating "etcd/healthcheck-client" certificate and key controlplane: [certs] Generating "apiserver-etcd-client" certificate and key controlplane: I0414 23:16:26.533376 39370 certs.go:78] creating new public/private key files for signing service account users controlplane: [certs] Generating "sa" key and public key controlplane: [kubeconfig] Using kubeconfig folder "/etc/kubernetes" controlplane: I0414 23:16:26.631048 39370 kubeconfig.go:103] creating kubeconfig file for admin.conf controlplane: [kubeconfig] Writing "admin.conf" kubeconfig file controlplane: I0414 23:16:26.805805 39370 kubeconfig.go:103] creating kubeconfig file for kubelet.conf controlplane: [kubeconfig] Writing "kubelet.conf" kubeconfig file controlplane: I0414 23:16:26.932629 39370 kubeconfig.go:103] creating kubeconfig file for controller-manager.conf controlplane: [kubeconfig] Writing "controller-manager.conf" kubeconfig file controlplane: I0414 23:16:27.238627 39370 kubeconfig.go:103] creating kubeconfig file for scheduler.conf controlplane: [kubeconfig] Writing "scheduler.conf" kubeconfig file controlplane: I0414 23:16:27.630921 39370 kubelet.go:67] Stopping the kubelet controlplane: [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" controlplane: [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" controlplane: [kubelet-start] Starting the kubelet controlplane: [control-plane] Using manifest folder "/etc/kubernetes/manifests" controlplane: [control-plane] Creating static Pod manifest for "kube-apiserver" controlplane: I0414 23:16:27.873591 39370 manifests.go:99] [control-plane] getting StaticPodSpecs controlplane: I0414 23:16:27.873958 39370 certs.go:519] validating certificate period for CA certificate controlplane: I0414 23:16:27.874222 39370 manifests.go:125] [control-plane] adding volume "ca-certs" for component "kube-apiserver" controlplane: I0414 23:16:27.874230 39370 manifests.go:125] [control-plane] adding volume "etc-ca-certificates" for component "kube-apiserver" controlplane: I0414 23:16:27.874236 39370 manifests.go:125] [control-plane] adding volume "etc-pki" for component "kube-apiserver" controlplane: I0414 23:16:27.874240 39370 manifests.go:125] [control-plane] adding volume "k8s-certs" for component "kube-apiserver" controlplane: I0414 23:16:27.874243 39370 manifests.go:125] [control-plane] adding volume "usr-local-share-ca-certificates" for component "kube-apiserver" controlplane: I0414 23:16:27.874247 39370 manifests.go:125] [control-plane] adding volume "usr-share-ca-certificates" for component "kube-apiserver" controlplane: [control-plane] Creating static Pod manifest for "kube-controller-manager" controlplane: I0414 23:16:27.876587 39370 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-apiserver" to "/etc/kubernetes/manifests/kube-apiserver.yaml" controlplane: I0414 23:16:27.876687 39370 manifests.go:99] [control-plane] getting StaticPodSpecs controlplane: I0414 23:16:27.876859 39370 manifests.go:125] [control-plane] adding volume "ca-certs" for component "kube-controller-manager" controlplane: I0414 23:16:27.876946 39370 manifests.go:125] [control-plane] adding volume "etc-ca-certificates" for component "kube-controller-manager" controlplane: I0414 23:16:27.876952 39370 manifests.go:125] [control-plane] adding volume "etc-pki" for component "kube-controller-manager" controlplane: I0414 23:16:27.876955 39370 manifests.go:125] [control-plane] adding volume "flexvolume-dir" for component "kube-controller-manager" controlplane: I0414 23:16:27.876959 39370 manifests.go:125] [control-plane] adding volume "k8s-certs" for component "kube-controller-manager" controlplane: I0414 23:16:27.876963 39370 manifests.go:125] [control-plane] adding volume "kubeconfig" for component "kube-controller-manager" controlplane: I0414 23:16:27.876966 39370 manifests.go:125] [control-plane] adding volume "usr-local-share-ca-certificates" for component "kube-controller-manager" controlplane: I0414 23:16:27.876970 39370 manifests.go:125] [control-plane] adding volume "usr-share-ca-certificates" for component "kube-controller-manager" controlplane: [control-plane] Creating static Pod manifest for "kube-scheduler" controlplane: I0414 23:16:27.877640 39370 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-controller-manager" to "/etc/kubernetes/manifests/kube-controller-manager.yaml" controlplane: I0414 23:16:27.877729 39370 manifests.go:99] [control-plane] getting StaticPodSpecs controlplane: I0414 23:16:27.877878 39370 manifests.go:125] [control-plane] adding volume "kubeconfig" for component "kube-scheduler" controlplane: [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests" controlplane: I0414 23:16:27.878217 39370 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-scheduler" to "/etc/kubernetes/manifests/kube-scheduler.yaml" controlplane: W0414 23:16:27.878404 39370 images.go:80] could not find officially supported version of etcd for Kubernetes v1.27.1, falling back to the nearest etcd version (3.5.7-0) controlplane: I0414 23:16:27.880230 39370 local.go:65] [etcd] wrote Static Pod manifest for a local etcd member to "/etc/kubernetes/manifests/etcd.yaml" controlplane: I0414 23:16:27.880334 39370 waitcontrolplane.go:83] [wait-control-plane] Waiting for the API server to be healthy controlplane: I0414 23:16:27.880817 39370 loader.go:373] Config loaded from file: /etc/kubernetes/admin.conf controlplane: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s controlplane: I0414 23:16:27.886011 39370 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s in 3 milliseconds controlplane: I0414 23:16:28.566263 39370 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s in 3 milliseconds controlplane: I0414 23:16:28.896612 39370 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s in 0 milliseconds controlplane: I0414 23:16:29.387190 39370 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s in 0 milliseconds controlplane: I0414 23:16:29.896894 39370 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s in 1 milliseconds controlplane: I0414 23:16:30.387261 39370 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s in 0 milliseconds controlplane: I0414 23:16:30.889282 39370 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s in 1 milliseconds controlplane: I0414 23:16:31.389313 39370 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s in 1 milliseconds controlplane: I0414 23:16:31.891430 39370 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s in 0 milliseconds controlplane: I0414 23:16:32.387913 39370 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s in 0 milliseconds controlplane: I0414 23:16:32.894801 39370 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s in 0 milliseconds controlplane: I0414 23:16:33.388463 39370 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s in 0 milliseconds controlplane: I0414 23:16:33.886871 39370 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s in 0 milliseconds controlplane: I0414 23:16:37.287305 39370 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s 500 Internal Server Error in 2896 milliseconds controlplane: I0414 23:16:37.393718 39370 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s 500 Internal Server Error in 3 milliseconds controlplane: I0414 23:16:37.890627 39370 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s 500 Internal Server Error in 1 milliseconds controlplane: I0414 23:16:38.389841 39370 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s 500 Internal Server Error in 2 milliseconds controlplane: I0414 23:16:38.889063 39370 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s 500 Internal Server Error in 2 milliseconds controlplane: I0414 23:16:39.392045 39370 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s 200 OK in 3 milliseconds controlplane: [apiclient] All control plane components are healthy after 11.519086 seconds controlplane: I0414 23:16:39.400967 39370 uploadconfig.go:112] [upload-config] Uploading the kubeadm ClusterConfiguration to a ConfigMap controlplane: [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace controlplane: I0414 23:16:39.428439 39370 round_trippers.go:553] POST https://10.20.30.10:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s 201 Created in 13 milliseconds controlplane: I0414 23:16:39.454613 39370 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s 201 Created in 21 milliseconds controlplane: I0414 23:16:39.467576 39370 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s 201 Created in 12 milliseconds controlplane: I0414 23:16:39.467778 39370 uploadconfig.go:126] [upload-config] Uploading the kubelet component config to a ConfigMap controlplane: [kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster controlplane: I0414 23:16:39.494008 39370 round_trippers.go:553] POST https://10.20.30.10:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s 201 Created in 19 milliseconds controlplane: I0414 23:16:39.506081 39370 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s 201 Created in 11 milliseconds controlplane: I0414 23:16:39.517154 39370 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s 201 Created in 9 milliseconds controlplane: I0414 23:16:39.519209 39370 uploadconfig.go:131] [upload-config] Preserving the CRISocket information for the control-plane node controlplane: I0414 23:16:39.519274 39370 patchnode.go:31] [patchnode] Uploading the CRI Socket information "unix:///var/run/containerd/containerd.sock" to the Node API object "controlplane" as an annotation controlplane: I0414 23:16:40.024289 39370 round_trippers.go:553] GET https://10.20.30.10:6443/api/v1/nodes/controlplane?timeout=10s 200 OK in 3 milliseconds controlplane: I0414 23:16:40.037395 39370 round_trippers.go:553] PATCH https://10.20.30.10:6443/api/v1/nodes/controlplane?timeout=10s 200 OK in 11 milliseconds controlplane: [upload-certs] Skipping phase. Please see --upload-certs controlplane: [mark-control-plane] Marking the node controlplane as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers] controlplane: [mark-control-plane] Marking the node controlplane as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule] controlplane: I0414 23:16:40.542025 39370 round_trippers.go:553] GET https://10.20.30.10:6443/api/v1/nodes/controlplane?timeout=10s 200 OK in 2 milliseconds controlplane: I0414 23:16:40.556075 39370 round_trippers.go:553] PATCH https://10.20.30.10:6443/api/v1/nodes/controlplane?timeout=10s 200 OK in 8 milliseconds controlplane: [bootstrap-token] Using token: i43qfo.gngtf54bcblctjl1 controlplane: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles controlplane: I0414 23:16:40.562749 39370 round_trippers.go:553] GET https://10.20.30.10:6443/api/v1/namespaces/kube-system/secrets/bootstrap-token-i43qfo?timeout=10s 404 Not Found in 3 milliseconds controlplane: I0414 23:16:40.570813 39370 round_trippers.go:553] POST https://10.20.30.10:6443/api/v1/namespaces/kube-system/secrets?timeout=10s 201 Created in 7 milliseconds controlplane: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes controlplane: I0414 23:16:40.580257 39370 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/clusterroles?timeout=10s 201 Created in 6 milliseconds controlplane: I0414 23:16:40.625188 39370 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 44 milliseconds controlplane: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials controlplane: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token controlplane: I0414 23:16:40.646666 39370 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 21 milliseconds controlplane: I0414 23:16:40.651020 39370 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 4 milliseconds controlplane: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster controlplane: I0414 23:16:40.661438 39370 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 4 milliseconds controlplane: [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace controlplane: I0414 23:16:40.661938 39370 clusterinfo.go:47] [bootstrap-token] loading admin kubeconfig controlplane: I0414 23:16:40.662848 39370 loader.go:373] Config loaded from file: /etc/kubernetes/admin.conf controlplane: I0414 23:16:40.662860 39370 clusterinfo.go:58] [bootstrap-token] copying the cluster from admin.conf to the bootstrap kubeconfig controlplane: I0414 23:16:40.663214 39370 clusterinfo.go:70] [bootstrap-token] creating/updating ConfigMap in kube-public namespace controlplane: I0414 23:16:40.669376 39370 round_trippers.go:553] POST https://10.20.30.10:6443/api/v1/namespaces/kube-public/configmaps?timeout=10s 201 Created in 6 milliseconds controlplane: I0414 23:16:40.669490 39370 clusterinfo.go:84] creating the RBAC rules for exposing the cluster-info ConfigMap in the kube-public namespace controlplane: I0414 23:16:40.674743 39370 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-public/roles?timeout=10s 201 Created in 5 milliseconds controlplane: I0414 23:16:40.684103 39370 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-public/rolebindings?timeout=10s 201 Created in 8 milliseconds controlplane: [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key controlplane: I0414 23:16:40.684318 39370 kubeletfinalize.go:90] [kubelet-finalize] Assuming that kubelet client certificate rotation is enabled: found "/var/lib/kubelet/pki/kubelet-client-current.pem" controlplane: I0414 23:16:40.686009 39370 loader.go:373] Config loaded from file: /etc/kubernetes/kubelet.conf controlplane: I0414 23:16:40.689228 39370 kubeletfinalize.go:134] [kubelet-finalize] Restarting the kubelet to enable client certificate rotation controlplane: I0414 23:16:41.104822 39370 round_trippers.go:553] GET https://10.20.30.10:6443/apis/apps/v1/namespaces/kube-system/deployments?labelSelector=k8s-app%3Dkube-dns 200 OK in 4 milliseconds controlplane: I0414 23:16:41.109607 39370 round_trippers.go:553] GET https://10.20.30.10:6443/api/v1/namespaces/kube-system/configmaps/coredns?timeout=10s 404 Not Found in 2 milliseconds controlplane: I0414 23:16:41.126408 39370 round_trippers.go:553] POST https://10.20.30.10:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s 201 Created in 16 milliseconds controlplane: I0414 23:16:41.137768 39370 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/clusterroles?timeout=10s 201 Created in 10 milliseconds controlplane: I0414 23:16:41.157511 39370 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 17 milliseconds controlplane: I0414 23:16:41.164430 39370 round_trippers.go:553] POST https://10.20.30.10:6443/api/v1/namespaces/kube-system/serviceaccounts?timeout=10s 201 Created in 5 milliseconds controlplane: I0414 23:16:41.181398 39370 round_trippers.go:553] POST https://10.20.30.10:6443/apis/apps/v1/namespaces/kube-system/deployments?timeout=10s 201 Created in 14 milliseconds controlplane: [addons] Applied essential addon: CoreDNS controlplane: I0414 23:16:41.218852 39370 round_trippers.go:553] POST https://10.20.30.10:6443/api/v1/namespaces/kube-system/services?timeout=10s 201 Created in 35 milliseconds controlplane: I0414 23:16:41.225137 39370 round_trippers.go:553] POST https://10.20.30.10:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s 201 Created in 5 milliseconds controlplane: I0414 23:16:41.241787 39370 round_trippers.go:553] POST https://10.20.30.10:6443/apis/apps/v1/namespaces/kube-system/daemonsets?timeout=10s 201 Created in 9 milliseconds controlplane: I0414 23:16:41.247207 39370 round_trippers.go:553] POST https://10.20.30.10:6443/api/v1/namespaces/kube-system/serviceaccounts?timeout=10s 201 Created in 4 milliseconds controlplane: I0414 23:16:41.267941 39370 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 20 milliseconds controlplane: I0414 23:16:41.273898 39370 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s 201 Created in 5 milliseconds controlplane: I0414 23:16:41.283444 39370 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s 201 Created in 9 milliseconds controlplane: [addons] Applied essential addon: kube-proxy controlplane: controlplane: Your Kubernetes control-plane has initialized successfully! controlplane: controlplane: To start using your cluster, you need to run the following as a regular user: controlplane: controlplane: mkdir -p $HOME/.kube controlplane: sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config controlplane: sudo chown $(id -u):$(id -g) $HOME/.kube/config controlplane: controlplane: Alternatively, if you are the root user, you can run: controlplane: controlplane: export KUBECONFIG=/etc/kubernetes/admin.conf controlplane: controlplane: You should now deploy a pod network to the cluster. controlplane: Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: controlplane: https://kubernetes.io/docs/concepts/cluster-administration/addons/ controlplane: controlplane: Then you can join any number of worker nodes by running the following on each as root: controlplane: controlplane: kubeadm join 10.20.30.10:6443 --token i43qfo.gngtf54bcblctjl1 \ controlplane: --discovery-token-ca-cert-hash sha256:09b2a1f4aad359f97a52ac0161673557181b66cc058953c0befaf6a6111a6432 controlplane: I0414 23:16:41.285798 39370 loader.go:373] Config loaded from file: /etc/kubernetes/admin.conf controlplane: I0414 23:16:41.286591 39370 loader.go:373] Config loaded from file: /etc/kubernetes/admin.conf controlplane: serviceaccount/kube-proxy-windows created controlplane: clusterrolebinding.rbac.authorization.k8s.io/node:kube-proxy created controlplane: clusterrolebinding.rbac.authorization.k8s.io/node:god2 created controlplane: clusterrolebinding.rbac.authorization.k8s.io/node:god3 created controlplane: clusterrolebinding.rbac.authorization.k8s.io/node:god4 created controlplane: Testing controlplane nodes! controlplane: NAMESPACE NAME READY STATUS RESTARTS AGE controlplane: kube-system etcd-controlplane 0/1 Running 0 2s controlplane: kube-system kube-apiserver-controlplane 0/1 Running 0 2s controlplane: kube-system kube-controller-manager-controlplane 0/1 Running 0 2s controlplane: kube-system kube-scheduler-controlplane 0/1 Running 0 3s ==> controlplane: Running provisioner: shell... controlplane: Running: C:/Users/mateuszl/AppData/Local/Temp/vagrant-shell20230415-22620-1d40vaq.sh controlplane: running calico installer now with pod_cidr 100.244.0.0/16 controlplane: node/controlplane untainted controlplane: error: taint "node-role.kubernetes.io/master" not found controlplane: namespace/calico-system created controlplane: namespace/tigera-operator created controlplane: customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io created controlplane: customresourcedefinition.apiextensions.k8s.io/imagesets.operator.tigera.io created controlplane: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io created controlplane: customresourcedefinition.apiextensions.k8s.io/tigerastatuses.operator.tigera.io created controlplane: serviceaccount/tigera-operator created controlplane: clusterrole.rbac.authorization.k8s.io/tigera-operator created controlplane: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created controlplane: deployment.apps/tigera-operator created controlplane: --2023-04-14 23:16:48-- https://raw.githubusercontent.com/projectcalico/calico/v3.25.1/manifests/custom-resources.yaml controlplane: Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ... controlplane: Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected. controlplane: HTTP request sent, awaiting response... 200 OK controlplane: Length: 827 [text/plain] controlplane: Saving to: ‘trigera-custom-resource.yaml’ controlplane: controlplane: 0K 100% 45.5M=0s controlplane: controlplane: 2023-04-14 23:16:48 (45.5 MB/s) - ‘trigera-custom-resource.yaml’ saved [827/827] controlplane: controlplane: installation.operator.tigera.io/default created controlplane: apiserver.operator.tigera.io/default created controlplane: installation.operator.tigera.io/default patched controlplane: waiting 20s for calico pods... controlplane: --2023-04-14 23:17:10-- https://raw.githubusercontent.com/projectcalico/calico/v3.25.1/manifests/calico-windows-vxlan.yaml controlplane: Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ... controlplane: Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected. controlplane: HTTP request sent, awaiting response... 200 OK controlplane: Length: 4157 (4.1K) [text/plain] controlplane: Saving to: ‘calico-windows.yaml’ controlplane: controlplane: 0K .... 100% 16.6M=0s controlplane: controlplane: 2023-04-14 23:17:10 (16.6 MB/s) - ‘calico-windows.yaml’ saved [4157/4157] controlplane: controlplane: configmap/calico-windows-config created controlplane: daemonset.apps/calico-node-windows created controlplane: % Total % Received % Xferd Average Speed Time Time Time Current controlplane: Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 60.8M 100 60.8M 0 0 1494k 0 0:00:41 0:00:41 --:--:-- 961k controlplane: Successfully set StrictAffinity to: true controlplane: NAME READY STATUS RESTARTS AGE controlplane: calico-kube-controllers-789dc4c76b-f458q 0/1 Pending 0 44s controlplane: calico-node-2qgph 0/1 Init:1/2 0 44s controlplane: calico-typha-6fbc8d5c5d-fjh8s 0/1 ContainerCreating 0 44s controlplane: csi-node-driver-r4j49 0/2 ContainerCreating 0 44s *********** vagrant up first run done ~~~~ ENTERING WINDOWS BRINGUP LOOP *** cni: calico calico: 3.25.1; containerd: 1.7.0 Bringing machine 'winw1' up with 'virtualbox' provider... ==> winw1: Importing base box 'sig-windows-dev-tools/windows-2019'... ==> winw1: Matching MAC address for NAT networking... ==> winw1: Checking if box 'sig-windows-dev-tools/windows-2019' version '1.0' is up to date... ==> winw1: Setting the name of the VM: sig-windows-dev-tools_winw1_1681514340806_56034 ==> winw1: Fixed port collision for 3389 => 3389. Now on port 2200. ==> winw1: Fixed port collision for 22 => 2222. Now on port 2201. ==> winw1: Clearing any previously set network interfaces... ==> winw1: Preparing network interfaces based on configuration... winw1: Adapter 1: nat winw1: Adapter 2: hostonly ==> winw1: Forwarding ports... winw1: 3389 (guest) => 2200 (host) (adapter 1) winw1: 5985 (guest) => 55985 (host) (adapter 1) winw1: 5986 (guest) => 55986 (host) (adapter 1) winw1: 22 (guest) => 2201 (host) (adapter 1) ==> winw1: Running 'pre-boot' VM customizations... ==> winw1: Booting VM... ==> winw1: Waiting for machine to boot. This may take a few minutes... winw1: WinRM address: 127.0.0.1:55985 winw1: WinRM username: vagrant winw1: WinRM execution_time_limit: PT2H winw1: WinRM transport: negotiate ==> winw1: Machine booted and ready! [winw1] No Virtualbox Guest Additions installation found. Copy iso file C:\Program Files\Oracle\VirtualBox\VBoxGuestAdditions.iso into the box $env:TEMP/VBoxGuestAdditions.iso Attached : True BlockSize : 0 DevicePath : \\.\CDROM0 FileSize : 53049344 ImagePath : C:\Users\vagrant\AppData\Local\Temp/VBoxGuestAdditions.iso LogicalSectorSize : 2048 Number : 0 Size : 53049344 StorageType : 1 PSComputerName : Mounting Virtualbox Guest Additions ISO to: D Installing Virtualbox Guest Additions 7.0.6 - guest version is unknown VBoxCertUtil.exe: info: Successfully added 'vbox-sha1.cer' to the trusted publisher store VBoxCertUtil.exe: info: Successfully added 'vbox-sha1-root.cer' to the trusted publisher store VBoxCertUtil.exe: info: Successfully added 'vbox-sha1-timestamp-root.cer' to the trusted publisher store VBoxCertUtil.exe: info: Successfully added 'vbox-sha256.cer' to the trusted publisher store VBoxCertUtil.exe: info: Successfully added 'vbox-sha256-root.cer' to the trusted publisher store VBoxCertUtil.exe: info: Successfully added 'vbox-sha256-timestamp-root.cer' to the trusted publisher store VBoxCertUtil.exe: info: Successfully added 'vbox-sha1.cer' to the root store VBoxCertUtil.exe: info: Certificate 'vbox-sha1-root.cer' is already present in the Root store and was not re-added or updated. VBoxCertUtil.exe: info: Successfully added 'vbox-sha1-timestamp-root.cer' to the root store VBoxCertUtil.exe: info: Successfully added 'vbox-sha256.cer' to the root store VBoxCertUtil.exe: info: Certificate 'vbox-sha256-root.cer' is already present in the Root store and was not re-added or updated. VBoxCertUtil.exe: info: Successfully added 'vbox-sha256-timestamp-root.cer' to the root store Unmounting Virtualbox Guest Additions ISO from: D Attached : False BlockSize : 0 DevicePath : FileSize : 53049344 ImagePath : C:\Users\vagrant\AppData\Local\Temp/VBoxGuestAdditions.iso LogicalSectorSize : 2048 Number : Size : 53049344 StorageType : 1 PSComputerName : ==> winw1: Checking for guest additions in VM... ==> winw1: Setting hostname... ==> winw1: Waiting for machine to reboot... ==> winw1: Configuring and enabling network interfaces... ==> winw1: Mounting shared folders... winw1: C:/forked => F:/Kubernetes/sig-windows-dev-tools/forked winw1: C:/sync/shared => F:/Kubernetes/sig-windows-dev-tools/sync/shared winw1: C:/sync/windows => F:/Kubernetes/sig-windows-dev-tools/sync/windows ==> winw1: Running provisioner: shell... winw1: Running: sync/windows/0-containerd.ps1 as C:\tmp\vagrant-shell.ps1 winw1: Stopping ContainerD & Kubelet winw1: Downloading Calico using ContainerD - [calico: 3.25] [containerd: 1.7.0] winw1: Installing 7Zip winw1: VERBOSE: Using the provider 'PowerShellGet' for searching packages. winw1: VERBOSE: Using the provider 'NuGet' for searching packages. winw1: VERBOSE: Total package yield:'0' for the specified package '7Zip4PowerShell'. winw1: VERBOSE: The -Repository parameter was not specified. PowerShellGet will use all of the registered repositories. winw1: VERBOSE: Getting the provider object for the PackageManagement Provider 'NuGet'. winw1: VERBOSE: The specified Location is 'https://www.powershellgallery.com/api/v2' and PackageManagementProvider is 'NuGet'. winw1: VERBOSE: Searching repository 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='7Zip4PowerShell'' for winw1: ''. winw1: VERBOSE: Total package yield:'1' for the specified package '7Zip4PowerShell'. winw1: VERBOSE: Performing the operation "Install Package" on target "Package '7Zip4Powershell' version '2.3.0' from winw1: 'PSGallery'.". winw1: VERBOSE: The installation scope is specified to be 'CurrentUser'. winw1: VERBOSE: The specified module will be installed in 'C:\Users\vagrant\Documents\WindowsPowerShell\Modules'. winw1: VERBOSE: The specified Location is 'NuGet' and PackageManagementProvider is 'NuGet'. winw1: VERBOSE: Downloading module '7Zip4Powershell' with version '2.3.0' from the repository winw1: 'https://www.powershellgallery.com/api/v2'. winw1: VERBOSE: Searching repository 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='7Zip4Powershell'' for winw1: ''. winw1: VERBOSE: InstallPackage' - name='7Zip4Powershell', winw1: version='2.3.0',destination='C:\Users\vagrant\AppData\Local\Temp\2054763731' winw1: VERBOSE: DownloadPackage' - name='7Zip4Powershell', winw1: version='2.3.0',destination='C:\Users\vagrant\AppData\Local\Temp\2054763731\7Zip4Powershell\7Zip4Powershell.nupkg', winw1: uri='https://www.powershellgallery.com/api/v2/package/7Zip4Powershell/2.3.0' winw1: VERBOSE: Downloading 'https://www.powershellgallery.com/api/v2/package/7Zip4Powershell/2.3.0'. winw1: VERBOSE: Completed downloading 'https://www.powershellgallery.com/api/v2/package/7Zip4Powershell/2.3.0'. winw1: VERBOSE: Completed downloading '7Zip4Powershell'. winw1: VERBOSE: Hash for package '7Zip4Powershell' does not match hash provided from the server. winw1: VERBOSE: InstallPackageLocal' - name='7Zip4Powershell', winw1: version='2.3.0',destination='C:\Users\vagrant\AppData\Local\Temp\2054763731' winw1: VERBOSE: Catalog file '7Zip4Powershell.cat' is not found in the contents of the module '7Zip4Powershell' being winw1: installed. winw1: VERBOSE: Module '7Zip4Powershell' was installed successfully to path winw1: 'C:\Users\vagrant\Documents\WindowsPowerShell\Modules\7Zip4Powershell\2.3.0'. winw1: winw1: Name Version Source Summary winw1: ---- ------- ------ ------- winw1: 7Zip4Powershell 2.3.0 PSGallery Powershell module for creating and extracting 7-Zip... winw1: Getting ContainerD binaries winw1: Downloading https://github.com/containerd/containerd/releases/download/v1.7.0/containerd-1.7.0-windows-amd64.tar.gz to C:\Program Files\containerd\containerd.tar.gz winw1: x containerd.exe winw1: x ctr.exe winw1: x containerd-stress.exe winw1: x containerd-shim-runhcs-v1.exe winw1: Registering ContainerD as a service winw1: time="2023-04-14T16:37:22.206655700-07:00" level=fatal msg="The specified service already exists." winw1: Starting ContainerD service winw1: Done - please remember to add '--cri-socket "npipe:////./pipe/containerd-containerd"' to your kubeadm join command winw1: winw1: ==> winw1: Running provisioner: shell... winw1: Running: sync/windows/forked.ps1 as C:\tmp\vagrant-shell.ps1 winw1: winw1: winw1: Directory: C:\ winw1: winw1: winw1: Mode LastWriteTime Length Name winw1: ---- ------------- ------ ---- winw1: d----- 1/21/2022 3:44 AM k winw1: winw1: ==> winw1: Running provisioner: shell... winw1: Running: sync/shared/kubejoin.ps1 as C:\tmp\vagrant-shell.ps1 winw1: [preflight] Running pre-flight checks winw1: W0414 16:38:28.646150 3292 checks.go:1064] [preflight] WARNING: Couldn't create the interface used for talking to the container runtime: crictl is required by the container runtime: executable file not found in %PATH% winw1: [preflight] Reading configuration from the cluster... winw1: [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml' winw1: W0414 16:38:37.833267 3292 initconfiguration.go:120] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "npipe" to the "criSocket" with value "unix:///var/run/unknown.sock". Please update your configuration! winw1: W0414 16:38:37.845328 3292 utils.go:69] The recommended value for "authentication.x509.clientCAFile" in "KubeletConfiguration" is: \etc\kubernetes\pki\ca.crt; the provided value is: /etc/kubernetes/pki/ca.crt winw1: [kubelet-start] Writing kubelet configuration to file "\\var\\lib\\kubelet\\config.yaml" winw1: [kubelet-start] Writing kubelet environment file with flags to file "\\var\\lib\\kubelet\\kubeadm-flags.env" winw1: [kubelet-start] Starting the kubelet winw1: [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap... winw1: winw1: This node has joined the cluster: winw1: * Certificate signing request was sent to apiserver and a response was received. winw1: * The Kubelet was informed of the new secure connection details. winw1: winw1: Run 'kubectl get nodes' on the control-plane to see this node join the cluster. cni: calico cni: calico deployment.apps/nginx-deployment created service/nginx created deployment.apps/whoami-windows created service/whoami-windows created pod/netshoot created Connection to 127.0.0.1 closed. cni: calico deployment.apps/whoami-windows scaled Connection to 127.0.0.1 closed. cni: calico deployment.apps/whoami-windows scaled Connection to 127.0.0.1 closed. cni: calico timed out waiting for the condition on pods/whoami-windows-9d46bfd7-4clgt timed out waiting for the condition on pods/whoami-windows-9d46bfd7-krnbd timed out waiting for the condition on pods/whoami-windows-9d46bfd7-ntqg8 Connection to 127.0.0.1 closed. make: *** [Makefile:60: 3-smoke-test] Error 1 ```

3. Check status of nodes

Despite the make all above terminated with 3-smoke-test error the two nodes of the cluster are Ready

$ vagrant ssh controlplane
cni: calico
Last login: Fri Apr 14 23:39:48 2023 from 10.0.2.2
vagrant@controlplane:~$ kubectl get nodes
NAME           STATUS     ROLES           AGE   VERSION
controlplane   Ready      control-plane   23m   v1.27.1-1+95feac5269be09
winw1          NotReady   <none>          67s   v1.27.1-1+95feac5269be09
vagrant@controlplane:~$ kubectl get nodes
NAME           STATUS     ROLES           AGE     VERSION
controlplane   Ready      control-plane   24m     v1.27.1-1+95feac5269be09
winw1          NotReady   <none>          2m22s   v1.27.1-1+95feac5269be09
vagrant@controlplane:~$ kubectl get nodes
NAME           STATUS     ROLES           AGE     VERSION
controlplane   Ready      control-plane   29m     v1.27.1-1+95feac5269be09
winw1          NotReady   <none>          6m55s   v1.27.1-1+95feac5269be09
vagrant@controlplane:~$ kubectl get nodes
NAME           STATUS     ROLES           AGE   VERSION
controlplane   Ready      control-plane   34m   v1.27.1-1+95feac5269be09
winw1          NotReady   <none>          12m   v1.27.1-1+95feac5269be09
vagrant@controlplane:~$ kubectl get nodes
NAME           STATUS   ROLES           AGE   VERSION
controlplane   Ready    control-plane   35m   v1.27.1-1+95feac5269be09
winw1          Ready    <none>          13m   v1.27.1-1+95feac5269be09
vagrant@controlplane:~$ kubectl get nodes
NAME           STATUS   ROLES           AGE   VERSION
controlplane   Ready    control-plane   36m   v1.27.1-1+95feac5269be09
winw1          Ready    <none>          14m   v1.27.1-1+95feac5269be09

4. Check status of pods

At least one pod on winw1 node has status Running, i.e. calico-node-windows-bjstr.

vagrant@controlplane:~$ kubectl get pods -A -o wide
NAMESPACE          NAME                                       READY   STATUS              RESTARTS      AGE   IP              NODE           NOMINATED NODE   READINESS GATES
calico-apiserver   calico-apiserver-8bbdd5967-4dc4v           1/1     Running             0             33m   100.244.49.70   controlplane   <none>           <none>
calico-apiserver   calico-apiserver-8bbdd5967-fvrrm           1/1     Running             0             33m   100.244.49.69   controlplane   <none>           <none>
calico-system      calico-kube-controllers-789dc4c76b-f458q   1/1     Running             0             36m   100.244.49.67   controlplane   <none>           <none>
calico-system      calico-node-2qgph                          1/1     Running             0             36m   10.20.30.10     controlplane   <none>           <none>
calico-system      calico-node-windows-bjstr                  1/2     Running             1 (60s ago)   15m   10.20.30.11     winw1          <none>           <none>
calico-system      calico-typha-6fbc8d5c5d-fjh8s              1/1     Running             0             36m   10.20.30.10     controlplane   <none>           <none>
calico-system      csi-node-driver-r4j49                      2/2     Running             0             36m   100.244.49.68   controlplane   <none>           <none>
default            netshoot                                   1/1     Running             0             14m   100.244.49.72   controlplane   <none>           <none>
default            nginx-deployment-7f97bd64fb-h288q          1/1     Running             0             14m   100.244.49.71   controlplane   <none>           <none>
default            whoami-windows-9d46bfd7-4clgt              0/1     ContainerCreating   0             14m   <none>          winw1          <none>           <none>
default            whoami-windows-9d46bfd7-krnbd              0/1     ContainerCreating   0             14m   <none>          winw1          <none>           <none>
default            whoami-windows-9d46bfd7-ntqg8              0/1     ContainerCreating   0             14m   <none>          winw1          <none>           <none>
kube-system        coredns-5d78c9869d-xrpcd                   1/1     Running             0             37m   100.244.49.65   controlplane   <none>           <none>
kube-system        coredns-5d78c9869d-zmtft                   1/1     Running             0             37m   100.244.49.66   controlplane   <none>           <none>
kube-system        etcd-controlplane                          1/1     Running             0             37m   10.20.30.10     controlplane   <none>           <none>
kube-system        kube-apiserver-controlplane                1/1     Running             0             37m   10.20.30.10     controlplane   <none>           <none>
kube-system        kube-controller-manager-controlplane       1/1     Running             0             37m   10.20.30.10     controlplane   <none>           <none>
kube-system        kube-proxy-ms54m                           1/1     Running             0             37m   10.20.30.10     controlplane   <none>           <none>
kube-system        kube-scheduler-controlplane                1/1     Running             0             37m   10.20.30.10     controlplane   <none>           <none>
tigera-operator    tigera-operator-549d4f9bdb-g6k97           1/1     Running             0             37m   10.20.30.10     controlplane   <none>           <none>

5. Run kubectl on host

Download kubeconfig from controlplane node:

vagrant plugin install vagrant-scp
vagrant scp controlplane:~/.kube/config ./.kubeconfig-sig-windows-dev-tools

Access cluster resources

kubectl get nodes --kubeconfig=./.kubeconfig-sig-windows-dev-tools -o wide
kubectl get pods -A --kubeconfig=./.kubeconfig-sig-windows-dev-tools -o wide
jayunit100 commented 1 year ago

great news thanks ! so does that mean we should merge this?

mloskot commented 1 year ago

@jayunit100

so does that mean we should merge this?

It turned out to be two threads here:

  1. The PR itself which proposes just a clarification of the currently documented procedure. It does not propose any substantial change of the procedure itself. This is completed.

  2. The test of the overall currently documented procedure on Windows host which attempts I documented in https://github.com/kubernetes-sigs/sig-windows-dev-tools/pull/245#issuecomment-1509712635 This is still work in progress - I need to build more understanding of the whole setup. For example, here are some issues I need to address:

    • update of versions in my patch to variables.yaml above may turn out to be unncessary and even insufficient as there is Windows Server 2019 image already built with references of hard-wired versions of tools or tools already deployed.

    • I'm not sure why ctrctl.exe is not being found as the WS 2019 image should have got it deployed already https://github.com/kubernetes-sigs/sig-windows-dev-tools/blob/4cfbbba1acea3e3b3dce27da9a81e14b1a8c6a58/experiments/image-builder/overlays/ansible/roles/utilities/tasks/main.yml#L15-L18

      but I'm seeing

       winw1: [preflight] WARNING: Couldn't create the interface used for talking to the container runtime:
          crictl is required for container runtime: exec: "crictl": executable file not found in %PATH%
    • I'm observing that it would be good to increase the Windows boot timeout in the Vagrantfile with patch like this

        --- a/Vagrantfile
        +++ b/Vagrantfile
        @@ -62,6 +62,8 @@ Vagrant.configure(2) do |config|
             winw1.vm.box = "sig-windows-dev-tools/windows-2019"
             winw1.vm.box_version = "1.0"
        +    winw1.vm.boot_timeout = 600

      To sum this thread up, I will continue discovering details and will try to discuss and confirm them here and on #sig-windows Slack channel.

Meanwhile, IMHO, this PR can be merged now and regardless of the results of my tests of the whole procedure.

mloskot commented 1 year ago

Update: Success 2 (workstation DEV690)

Following my first attempt in https://github.com/kubernetes-sigs/sig-windows-dev-tools/pull/245#issuecomment-1509229142, I have managed to successfully create the two-node cluster on another machine and using default variables.yaml without any patches. The only patch I had to apply was larger winw1.vm.boot_timeout as I had been experiencing Vagrant timetous for the Windows VM.

0. Host Environment

1. Patches

Unlike in Successful-Run-1, this time no CPU and memory updates to variables.yaml.

Vagrantfile was patched to avoid Windows node boot timeouts:

diff --git a/Vagrantfile b/Vagrantfile
index 1cfc0fe..45b15c7 100644
--- a/Vagrantfile
+++ b/Vagrantfile
@@ -61,6 +61,7 @@ Vagrant.configure(2) do |config|
     winw1.vm.host_name = "winw1"
     winw1.vm.box = "sig-windows-dev-tools/windows-2019"
     winw1.vm.box_version = "1.0"
+    winw1.vm.boot_timeout = 600

2. Run make all

Inside WSL terminal (Ubuntu 22.04) run:

export VAGRANT=/mnt/c/HashiCorp/Vagrant/bin/vagrant.exe
cd sig-windows-dev-tools
make all
Full make all log ``` Welcome to Ubuntu 22.04.2 LTS (GNU/Linux 5.15.90.1-microsoft-standard-WSL2 x86_64) * Documentation: https://help.ubuntu.com * Management: https://landscape.canonical.com * Support: https://ubuntu.com/advantage This message is shown once a day. To disable it please create the /home/mloskot/.hushlogin file. Loading ~/.bashrc mloskot@DEV690:~$ cd /mnt/f/Kubernetes/sig-windows-dev-tools/ mloskot@DEV690:/mnt/f/Kubernetes/sig-windows-dev-tools$ make all cni: calico calico: 3.25.0; containerd: 1.6.15 ==> winw1: VM not created. Moving on... ==> controlplane: VM not created. Moving on... clean % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 24 100 24 0 0 51 0 --:--:-- --:--:-- --:--:-- 51 Using Kubernetes version v1.26.4-3-2d4a3e29be572e from upstream build % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 24 100 24 0 0 89 0 --:--:-- --:--:-- --:--:-- 89 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 113M 100 113M 0 0 7055k 0 0:00:16 0:00:16 --:--:-- 8512k % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 115M 100 115M 0 0 7436k 0 0:00:15 0:00:15 --:--:-- 8547k % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 48.4M 100 48.4M 0 0 5899k 0 0:00:08 0:00:08 --:--:-- 8464k % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 42.9M 100 42.9M 0 0 5534k 0 0:00:07 0:00:07 --:--:-- 8643k % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 45.1M 100 45.1M 0 0 5530k 0 0:00:08 0:00:08 --:--:-- 8285k % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 44.6M 100 44.6M 0 0 6284k 0 0:00:07 0:00:07 --:--:-- 8630k % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 46.4M 100 46.4M 0 0 5744k 0 0:00:08 0:00:08 --:--:-- 8199k % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 45.8M 100 45.8M 0 0 5978k 0 0:00:07 0:00:07 --:--:-- 8360k vagrant phase making mock kubejoin file to keep Vagrantfile happy in sync/shared ###################################### Retry vagrant up if the first time the windows node failed Starting the control plane ###################################### cni: calico calico: 3.25.0; containerd: 1.6.15 Bringing machine 'controlplane' up with 'virtualbox' provider... ==> controlplane: Importing base box 'roboxes/ubuntu2004'... ==> controlplane: Matching MAC address for NAT networking... ==> controlplane: Checking if box 'roboxes/ubuntu2004' version '4.2.14' is up to date... ==> controlplane: Setting the name of the VM: sig-windows-dev-tools_controlplane_1681640403294_46202 ==> controlplane: Clearing any previously set network interfaces... ==> controlplane: Preparing network interfaces based on configuration... controlplane: Adapter 1: nat controlplane: Adapter 2: hostonly ==> controlplane: Forwarding ports... controlplane: 22 (guest) => 2222 (host) (adapter 1) ==> controlplane: Running 'pre-boot' VM customizations... ==> controlplane: Booting VM... ==> controlplane: Waiting for machine to boot. This may take a few minutes... controlplane: SSH address: 127.0.0.1:2222 controlplane: SSH username: vagrant controlplane: SSH auth method: private key controlplane: Warning: Connection reset. Retrying... controlplane: controlplane: Vagrant insecure key detected. Vagrant will automatically replace controlplane: this with a newly generated keypair for better security. controlplane: controlplane: Inserting generated public key within guest... controlplane: Removing insecure key from the guest if it's present... controlplane: Key inserted! Disconnecting and reconnecting using new SSH key... ==> controlplane: Machine booted and ready! [controlplane] A Virtualbox Guest Additions installation was found but no tools to rebuild or start them. Reading package lists... Building dependency tree... Reading state information... Package 'virtualbox-guest-dkms' is not installed, so not removed Package 'virtualbox-guest-x11' is not installed, so not removed The following packages will be REMOVED: virtualbox-guest-utils* 0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded. After this operation, 6,070 kB disk space will be freed. (Reading database ... 111809 files and directories currently installed.) Removing virtualbox-guest-utils (6.1.38-dfsg-3~ubuntu1.20.04.1) ... Processing triggers for man-db (2.9.1-1) ... (Reading database ... 111796 files and directories currently installed.) Purging configuration files for virtualbox-guest-utils (6.1.38-dfsg-3~ubuntu1.20.04.1) ... Processing triggers for systemd (245.4-4ubuntu3.19) ... Reading package lists... Building dependency tree... Reading state information... linux-headers-5.4.0-139-generic is already the newest version (5.4.0-139.156). linux-headers-5.4.0-139-generic set to manually installed. The following additional packages will be installed: binutils binutils-common binutils-x86-64-linux-gnu cpp cpp-9 dctrl-tools dpkg-dev fakeroot g++ g++-9 gcc gcc-9 gcc-9-base libalgorithm-diff-perl libalgorithm-diff-xs-perl libalgorithm-merge-perl libasan5 libatomic1 libbinutils libc-dev-bin libc6-dev libcc1-0 libcrypt-dev libctf-nobfd0 libctf0 libdpkg-perl libfakeroot libfile-fcntllock-perl libgcc-9-dev libgomp1 libisl22 libitm1 liblsan0 libmpc3 libquadmath0 libstdc++-9-dev libtsan0 libubsan1 linux-libc-dev make manpages-dev Suggested packages: binutils-doc cpp-doc gcc-9-locales debtags menu debian-keyring g++-multilib g++-9-multilib gcc-9-doc gcc-multilib autoconf automake libtool flex bison gdb gcc-doc gcc-9-multilib glibc-doc bzr libstdc++-9-doc make-doc The following NEW packages will be installed: binutils binutils-common binutils-x86-64-linux-gnu build-essential cpp cpp-9 dctrl-tools dkms dpkg-dev fakeroot g++ g++-9 gcc gcc-9 gcc-9-base libalgorithm-diff-perl libalgorithm-diff-xs-perl libalgorithm-merge-perl libasan5 libatomic1 libbinutils libc-dev-bin libc6-dev libcc1-0 libcrypt-dev libctf-nobfd0 libctf0 libdpkg-perl libfakeroot libfile-fcntllock-perl libgcc-9-dev libgomp1 libisl22 libitm1 liblsan0 libmpc3 libquadmath0 libstdc++-9-dev libtsan0 libubsan1 linux-libc-dev make manpages-dev 0 upgraded, 43 newly installed, 0 to remove and 0 not upgraded. Need to get 45.5 MB of archives. After this operation, 203 MB of additional disk space will be used. Get:1 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 gcc-9-base amd64 9.4.0-1ubuntu1~20.04.1 [19.4 kB] Get:2 http://us.archive.ubuntu.com/ubuntu focal/main amd64 libisl22 amd64 0.22.1-1 [592 kB] Get:3 http://us.archive.ubuntu.com/ubuntu focal/main amd64 libmpc3 amd64 1.1.0-1 [40.8 kB] Get:4 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 cpp-9 amd64 9.4.0-1ubuntu1~20.04.1 [7,500 kB] Get:5 http://us.archive.ubuntu.com/ubuntu focal/main amd64 cpp amd64 4:9.3.0-1ubuntu2 [27.6 kB] Get:6 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 libcc1-0 amd64 10.3.0-1ubuntu1~20.04 [48.8 kB] Get:7 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 binutils-common amd64 2.34-6ubuntu1.4 [207 kB] Get:8 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 libbinutils amd64 2.34-6ubuntu1.4 [474 kB] Get:9 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 libctf-nobfd0 amd64 2.34-6ubuntu1.4 [47.2 kB] Get:10 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 libctf0 amd64 2.34-6ubuntu1.4 [46.6 kB] Get:11 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 binutils-x86-64-linux-gnu amd64 2.34-6ubuntu1.4 [1,613 kB] Get:12 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 binutils amd64 2.34-6ubuntu1.4 [3,380 B] Get:13 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 libgomp1 amd64 10.3.0-1ubuntu1~20.04 [102 kB] Get:14 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 libitm1 amd64 10.3.0-1ubuntu1~20.04 [26.2 kB] Get:15 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 libatomic1 amd64 10.3.0-1ubuntu1~20.04 [9,284 B] Get:16 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 libasan5 amd64 9.4.0-1ubuntu1~20.04.1 [2,751 kB] Get:17 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 liblsan0 amd64 10.3.0-1ubuntu1~20.04 [835 kB] Get:18 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 libtsan0 amd64 10.3.0-1ubuntu1~20.04 [2,009 kB] Get:19 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 libubsan1 amd64 10.3.0-1ubuntu1~20.04 [784 kB] Get:20 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 libquadmath0 amd64 10.3.0-1ubuntu1~20.04 [146 kB] Get:21 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 libgcc-9-dev amd64 9.4.0-1ubuntu1~20.04.1 [2,359 kB] Get:22 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 gcc-9 amd64 9.4.0-1ubuntu1~20.04.1 [8,274 kB] Get:23 http://us.archive.ubuntu.com/ubuntu focal/main amd64 gcc amd64 4:9.3.0-1ubuntu2 [5,208 B] Get:24 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 libdpkg-perl all 1.19.7ubuntu3.2 [231 kB] Get:25 http://us.archive.ubuntu.com/ubuntu focal/main amd64 make amd64 4.2.1-1.2 [162 kB] Get:26 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 dpkg-dev all 1.19.7ubuntu3.2 [679 kB] Get:27 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 libc-dev-bin amd64 2.31-0ubuntu9.9 [71.8 kB] Ign:28 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-libc-dev amd64 5.4.0-139.156 Get:29 http://us.archive.ubuntu.com/ubuntu focal/main amd64 libcrypt-dev amd64 1:4.4.10-10ubuntu4 [104 kB] Get:30 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 libc6-dev amd64 2.31-0ubuntu9.9 [2,519 kB] Get:31 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 libstdc++-9-dev amd64 9.4.0-1ubuntu1~20.04.1 [1,722 kB] Get:32 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 g++-9 amd64 9.4.0-1ubuntu1~20.04.1 [8,420 kB] Get:33 http://us.archive.ubuntu.com/ubuntu focal/main amd64 g++ amd64 4:9.3.0-1ubuntu2 [1,604 B] Get:34 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 build-essential amd64 12.8ubuntu1.1 [4,664 B] Get:35 http://us.archive.ubuntu.com/ubuntu focal/main amd64 dctrl-tools amd64 2.24-3 [61.5 kB] Get:36 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 dkms all 2.8.1-5ubuntu2 [66.8 kB] Get:37 http://us.archive.ubuntu.com/ubuntu focal/main amd64 libfakeroot amd64 1.24-1 [25.7 kB] Get:38 http://us.archive.ubuntu.com/ubuntu focal/main amd64 fakeroot amd64 1.24-1 [62.6 kB] Get:39 http://us.archive.ubuntu.com/ubuntu focal/main amd64 libalgorithm-diff-perl all 1.19.03-2 [46.6 kB] Get:40 http://us.archive.ubuntu.com/ubuntu focal/main amd64 libalgorithm-diff-xs-perl amd64 0.04-6 [11.3 kB] Get:41 http://us.archive.ubuntu.com/ubuntu focal/main amd64 libalgorithm-merge-perl all 0.08-3 [12.0 kB] Get:42 http://us.archive.ubuntu.com/ubuntu focal/main amd64 libfile-fcntllock-perl amd64 0.22-3build4 [33.1 kB] Get:43 http://us.archive.ubuntu.com/ubuntu focal/main amd64 manpages-dev all 5.05-1 [2,266 kB] Err:28 http://security.ubuntu.com/ubuntu focal-updates/main amd64 linux-libc-dev amd64 5.4.0-139.156 404 Not Found [IP: 91.189.91.39 80] Fetched 44.4 MB in 13s (3,546 kB/s) E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/l/linux/linux-libc-dev_5.4.0-139.156_amd64.deb 404 Not Found [IP: 91.189.91.39 80] E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing? Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] Hit:2 http://us.archive.ubuntu.com/ubuntu focal InRelease Get:3 http://us.archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] Get:4 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [575 kB] Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,085 kB] Get:6 http://us.archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] Get:7 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [338 kB] Get:8 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [12.5 kB] Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,609 kB] Get:10 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,469 kB] Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [29.1 kB] Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [227 kB] Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [612 B] Get:14 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [821 kB] Get:15 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [591 kB] Get:16 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [165 kB] Get:17 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [17.6 kB] Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [22.9 kB] Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [516 B] Get:21 http://us.archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [807 kB] Get:22 http://us.archive.ubuntu.com/ubuntu focal-updates/main Translation-en [421 kB] Get:23 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [16.4 kB] Get:24 http://us.archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,717 kB] Get:25 http://us.archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [30.4 kB] Get:26 http://us.archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [242 kB] Get:27 http://us.archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [612 B] Get:28 http://us.archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [722 kB] Get:29 http://us.archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,047 kB] Get:30 http://us.archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [247 kB] Get:31 http://us.archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [24.2 kB] Get:32 http://us.archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [25.2 kB] Get:33 http://us.archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,436 B] Get:34 http://us.archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [592 B] Get:35 http://us.archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] Get:36 http://us.archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] Get:37 http://us.archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] Get:38 http://us.archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] Get:39 http://us.archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [24.9 kB] Get:40 http://us.archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] Fetched 14.7 MB in 5s (2,915 kB/s) Reading package lists... W: --force-yes is deprecated, use one of the options starting with --allow instead. Reading package lists... Building dependency tree... Reading state information... linux-headers-5.4.0-139-generic is already the newest version (5.4.0-139.156). linux-headers-5.4.0-139-generic set to manually installed. The following additional packages will be installed: binutils binutils-common binutils-x86-64-linux-gnu cpp cpp-9 dctrl-tools dpkg-dev fakeroot g++ g++-9 gcc gcc-9 gcc-9-base libalgorithm-diff-perl libalgorithm-diff-xs-perl libalgorithm-merge-perl libasan5 libatomic1 libbinutils libc-dev-bin libc6-dev libcc1-0 libcrypt-dev libctf-nobfd0 libctf0 libdpkg-perl libfakeroot libfile-fcntllock-perl libgcc-9-dev libgomp1 libisl22 libitm1 liblsan0 libmpc3 libquadmath0 libstdc++-9-dev libtsan0 libubsan1 linux-libc-dev make manpages-dev Suggested packages: binutils-doc cpp-doc gcc-9-locales debtags menu debian-keyring g++-multilib g++-9-multilib gcc-9-doc gcc-multilib autoconf automake libtool flex bison gdb gcc-doc gcc-9-multilib glibc-doc bzr libstdc++-9-doc make-doc The following NEW packages will be installed: binutils binutils-common binutils-x86-64-linux-gnu build-essential cpp cpp-9 dctrl-tools dkms dpkg-dev fakeroot g++ g++-9 gcc gcc-9 gcc-9-base libalgorithm-diff-perl libalgorithm-diff-xs-perl libalgorithm-merge-perl libasan5 libatomic1 libbinutils libc-dev-bin libc6-dev libcc1-0 libcrypt-dev libctf-nobfd0 libctf0 libdpkg-perl libfakeroot libfile-fcntllock-perl libgcc-9-dev libgomp1 libisl22 libitm1 liblsan0 libmpc3 libquadmath0 libstdc++-9-dev libtsan0 libubsan1 linux-libc-dev make manpages-dev 0 upgraded, 43 newly installed, 0 to remove and 70 not upgraded. Need to get 1,127 kB/45.5 MB of archives. After this operation, 203 MB of additional disk space will be used. Get:1 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-libc-dev amd64 5.4.0-146.163 [1,127 kB] Fetched 1,127 kB in 1s (1,314 kB/s) Selecting previously unselected package gcc-9-base:amd64. (Reading database ... 111795 files and directories currently installed.) Preparing to unpack .../00-gcc-9-base_9.4.0-1ubuntu1~20.04.1_amd64.deb ... Unpacking gcc-9-base:amd64 (9.4.0-1ubuntu1~20.04.1) ... Selecting previously unselected package libisl22:amd64. Preparing to unpack .../01-libisl22_0.22.1-1_amd64.deb ... Unpacking libisl22:amd64 (0.22.1-1) ... Selecting previously unselected package libmpc3:amd64. Preparing to unpack .../02-libmpc3_1.1.0-1_amd64.deb ... Unpacking libmpc3:amd64 (1.1.0-1) ... Selecting previously unselected package cpp-9. Preparing to unpack .../03-cpp-9_9.4.0-1ubuntu1~20.04.1_amd64.deb ... Unpacking cpp-9 (9.4.0-1ubuntu1~20.04.1) ... Selecting previously unselected package cpp. Preparing to unpack .../04-cpp_4%3a9.3.0-1ubuntu2_amd64.deb ... Unpacking cpp (4:9.3.0-1ubuntu2) ... Selecting previously unselected package libcc1-0:amd64. Preparing to unpack .../05-libcc1-0_10.3.0-1ubuntu1~20.04_amd64.deb ... Unpacking libcc1-0:amd64 (10.3.0-1ubuntu1~20.04) ... Selecting previously unselected package binutils-common:amd64. Preparing to unpack .../06-binutils-common_2.34-6ubuntu1.4_amd64.deb ... Unpacking binutils-common:amd64 (2.34-6ubuntu1.4) ... Selecting previously unselected package libbinutils:amd64. Preparing to unpack .../07-libbinutils_2.34-6ubuntu1.4_amd64.deb ... Unpacking libbinutils:amd64 (2.34-6ubuntu1.4) ... Selecting previously unselected package libctf-nobfd0:amd64. Preparing to unpack .../08-libctf-nobfd0_2.34-6ubuntu1.4_amd64.deb ... Unpacking libctf-nobfd0:amd64 (2.34-6ubuntu1.4) ... Selecting previously unselected package libctf0:amd64. Preparing to unpack .../09-libctf0_2.34-6ubuntu1.4_amd64.deb ... Unpacking libctf0:amd64 (2.34-6ubuntu1.4) ... Selecting previously unselected package binutils-x86-64-linux-gnu. Preparing to unpack .../10-binutils-x86-64-linux-gnu_2.34-6ubuntu1.4_amd64.deb ... Unpacking binutils-x86-64-linux-gnu (2.34-6ubuntu1.4) ... Selecting previously unselected package binutils. Preparing to unpack .../11-binutils_2.34-6ubuntu1.4_amd64.deb ... Unpacking binutils (2.34-6ubuntu1.4) ... Selecting previously unselected package libgomp1:amd64. Preparing to unpack .../12-libgomp1_10.3.0-1ubuntu1~20.04_amd64.deb ... Unpacking libgomp1:amd64 (10.3.0-1ubuntu1~20.04) ... Selecting previously unselected package libitm1:amd64. Preparing to unpack .../13-libitm1_10.3.0-1ubuntu1~20.04_amd64.deb ... Unpacking libitm1:amd64 (10.3.0-1ubuntu1~20.04) ... Selecting previously unselected package libatomic1:amd64. Preparing to unpack .../14-libatomic1_10.3.0-1ubuntu1~20.04_amd64.deb ... Unpacking libatomic1:amd64 (10.3.0-1ubuntu1~20.04) ... Selecting previously unselected package libasan5:amd64. Preparing to unpack .../15-libasan5_9.4.0-1ubuntu1~20.04.1_amd64.deb ... Unpacking libasan5:amd64 (9.4.0-1ubuntu1~20.04.1) ... Selecting previously unselected package liblsan0:amd64. Preparing to unpack .../16-liblsan0_10.3.0-1ubuntu1~20.04_amd64.deb ... Unpacking liblsan0:amd64 (10.3.0-1ubuntu1~20.04) ... Selecting previously unselected package libtsan0:amd64. Preparing to unpack .../17-libtsan0_10.3.0-1ubuntu1~20.04_amd64.deb ... Unpacking libtsan0:amd64 (10.3.0-1ubuntu1~20.04) ... Selecting previously unselected package libubsan1:amd64. Preparing to unpack .../18-libubsan1_10.3.0-1ubuntu1~20.04_amd64.deb ... Unpacking libubsan1:amd64 (10.3.0-1ubuntu1~20.04) ... Selecting previously unselected package libquadmath0:amd64. Preparing to unpack .../19-libquadmath0_10.3.0-1ubuntu1~20.04_amd64.deb ... Unpacking libquadmath0:amd64 (10.3.0-1ubuntu1~20.04) ... Selecting previously unselected package libgcc-9-dev:amd64. Preparing to unpack .../20-libgcc-9-dev_9.4.0-1ubuntu1~20.04.1_amd64.deb ... Unpacking libgcc-9-dev:amd64 (9.4.0-1ubuntu1~20.04.1) ... Selecting previously unselected package gcc-9. Preparing to unpack .../21-gcc-9_9.4.0-1ubuntu1~20.04.1_amd64.deb ... Unpacking gcc-9 (9.4.0-1ubuntu1~20.04.1) ... Selecting previously unselected package gcc. Preparing to unpack .../22-gcc_4%3a9.3.0-1ubuntu2_amd64.deb ... Unpacking gcc (4:9.3.0-1ubuntu2) ... Selecting previously unselected package libdpkg-perl. Preparing to unpack .../23-libdpkg-perl_1.19.7ubuntu3.2_all.deb ... Unpacking libdpkg-perl (1.19.7ubuntu3.2) ... Selecting previously unselected package make. Preparing to unpack .../24-make_4.2.1-1.2_amd64.deb ... Unpacking make (4.2.1-1.2) ... Selecting previously unselected package dpkg-dev. Preparing to unpack .../25-dpkg-dev_1.19.7ubuntu3.2_all.deb ... Unpacking dpkg-dev (1.19.7ubuntu3.2) ... Selecting previously unselected package libc-dev-bin. Preparing to unpack .../26-libc-dev-bin_2.31-0ubuntu9.9_amd64.deb ... Unpacking libc-dev-bin (2.31-0ubuntu9.9) ... Selecting previously unselected package linux-libc-dev:amd64. Preparing to unpack .../27-linux-libc-dev_5.4.0-146.163_amd64.deb ... Unpacking linux-libc-dev:amd64 (5.4.0-146.163) ... Selecting previously unselected package libcrypt-dev:amd64. Preparing to unpack .../28-libcrypt-dev_1%3a4.4.10-10ubuntu4_amd64.deb ... Unpacking libcrypt-dev:amd64 (1:4.4.10-10ubuntu4) ... Selecting previously unselected package libc6-dev:amd64. Preparing to unpack .../29-libc6-dev_2.31-0ubuntu9.9_amd64.deb ... Unpacking libc6-dev:amd64 (2.31-0ubuntu9.9) ... Selecting previously unselected package libstdc++-9-dev:amd64. Preparing to unpack .../30-libstdc++-9-dev_9.4.0-1ubuntu1~20.04.1_amd64.deb ... Unpacking libstdc++-9-dev:amd64 (9.4.0-1ubuntu1~20.04.1) ... Selecting previously unselected package g++-9. Preparing to unpack .../31-g++-9_9.4.0-1ubuntu1~20.04.1_amd64.deb ... Unpacking g++-9 (9.4.0-1ubuntu1~20.04.1) ... Selecting previously unselected package g++. Preparing to unpack .../32-g++_4%3a9.3.0-1ubuntu2_amd64.deb ... Unpacking g++ (4:9.3.0-1ubuntu2) ... Selecting previously unselected package build-essential. Preparing to unpack .../33-build-essential_12.8ubuntu1.1_amd64.deb ... Unpacking build-essential (12.8ubuntu1.1) ... Selecting previously unselected package dctrl-tools. Preparing to unpack .../34-dctrl-tools_2.24-3_amd64.deb ... Unpacking dctrl-tools (2.24-3) ... Selecting previously unselected package dkms. Preparing to unpack .../35-dkms_2.8.1-5ubuntu2_all.deb ... Unpacking dkms (2.8.1-5ubuntu2) ... Selecting previously unselected package libfakeroot:amd64. Preparing to unpack .../36-libfakeroot_1.24-1_amd64.deb ... Unpacking libfakeroot:amd64 (1.24-1) ... Selecting previously unselected package fakeroot. Preparing to unpack .../37-fakeroot_1.24-1_amd64.deb ... Unpacking fakeroot (1.24-1) ... Selecting previously unselected package libalgorithm-diff-perl. Preparing to unpack .../38-libalgorithm-diff-perl_1.19.03-2_all.deb ... Unpacking libalgorithm-diff-perl (1.19.03-2) ... Selecting previously unselected package libalgorithm-diff-xs-perl. Preparing to unpack .../39-libalgorithm-diff-xs-perl_0.04-6_amd64.deb ... Unpacking libalgorithm-diff-xs-perl (0.04-6) ... Selecting previously unselected package libalgorithm-merge-perl. Preparing to unpack .../40-libalgorithm-merge-perl_0.08-3_all.deb ... Unpacking libalgorithm-merge-perl (0.08-3) ... Selecting previously unselected package libfile-fcntllock-perl. Preparing to unpack .../41-libfile-fcntllock-perl_0.22-3build4_amd64.deb ... Unpacking libfile-fcntllock-perl (0.22-3build4) ... Selecting previously unselected package manpages-dev. Preparing to unpack .../42-manpages-dev_5.05-1_all.deb ... Unpacking manpages-dev (5.05-1) ... Setting up manpages-dev (5.05-1) ... Setting up libfile-fcntllock-perl (0.22-3build4) ... Setting up libalgorithm-diff-perl (1.19.03-2) ... Setting up binutils-common:amd64 (2.34-6ubuntu1.4) ... Setting up linux-libc-dev:amd64 (5.4.0-146.163) ... Setting up libctf-nobfd0:amd64 (2.34-6ubuntu1.4) ... Setting up libgomp1:amd64 (10.3.0-1ubuntu1~20.04) ... Setting up libfakeroot:amd64 (1.24-1) ... Setting up fakeroot (1.24-1) ... update-alternatives: using /usr/bin/fakeroot-sysv to provide /usr/bin/fakeroot (fakeroot) in auto mode Setting up make (4.2.1-1.2) ... Setting up libquadmath0:amd64 (10.3.0-1ubuntu1~20.04) ... Setting up libmpc3:amd64 (1.1.0-1) ... Setting up libatomic1:amd64 (10.3.0-1ubuntu1~20.04) ... Setting up libdpkg-perl (1.19.7ubuntu3.2) ... Setting up libubsan1:amd64 (10.3.0-1ubuntu1~20.04) ... Setting up libcrypt-dev:amd64 (1:4.4.10-10ubuntu4) ... Setting up libisl22:amd64 (0.22.1-1) ... Setting up libbinutils:amd64 (2.34-6ubuntu1.4) ... Setting up libc-dev-bin (2.31-0ubuntu9.9) ... Setting up libalgorithm-diff-xs-perl (0.04-6) ... Setting up libcc1-0:amd64 (10.3.0-1ubuntu1~20.04) ... Setting up liblsan0:amd64 (10.3.0-1ubuntu1~20.04) ... Setting up dctrl-tools (2.24-3) ... Setting up libitm1:amd64 (10.3.0-1ubuntu1~20.04) ... Setting up gcc-9-base:amd64 (9.4.0-1ubuntu1~20.04.1) ... Setting up libalgorithm-merge-perl (0.08-3) ... Setting up libtsan0:amd64 (10.3.0-1ubuntu1~20.04) ... Setting up libctf0:amd64 (2.34-6ubuntu1.4) ... Setting up libasan5:amd64 (9.4.0-1ubuntu1~20.04.1) ... Setting up cpp-9 (9.4.0-1ubuntu1~20.04.1) ... Setting up libc6-dev:amd64 (2.31-0ubuntu9.9) ... Setting up binutils-x86-64-linux-gnu (2.34-6ubuntu1.4) ... Setting up binutils (2.34-6ubuntu1.4) ... Setting up dpkg-dev (1.19.7ubuntu3.2) ... Setting up libgcc-9-dev:amd64 (9.4.0-1ubuntu1~20.04.1) ... Setting up cpp (4:9.3.0-1ubuntu2) ... Setting up gcc-9 (9.4.0-1ubuntu1~20.04.1) ... Setting up libstdc++-9-dev:amd64 (9.4.0-1ubuntu1~20.04.1) ... Setting up gcc (4:9.3.0-1ubuntu2) ... Setting up dkms (2.8.1-5ubuntu2) ... Setting up g++-9 (9.4.0-1ubuntu1~20.04.1) ... Setting up g++ (4:9.3.0-1ubuntu2) ... update-alternatives: using /usr/bin/g++ to provide /usr/bin/c++ (c++) in auto mode Setting up build-essential (12.8ubuntu1.1) ... Processing triggers for man-db (2.9.1-1) ... Processing triggers for libc-bin (2.31-0ubuntu9.9) ... Copy iso file C:\Program Files\Oracle\VirtualBox\VBoxGuestAdditions.iso into the box /tmp/VBoxGuestAdditions.iso Mounting Virtualbox Guest Additions ISO to: /mnt mount: /mnt: WARNING: device write-protected, mounted read-only. Installing Virtualbox Guest Additions 7.0.6 - guest version is 6.1.38 Verifying archive integrity... 100% MD5 checksums are OK. All good. Uncompressing VirtualBox 7.0.6 Guest Additions for Linux 100% VirtualBox Guest Additions installer /opt/VBoxGuestAdditions-7.0.6/bin/VBoxClient: error while loading shared libraries: libXt.so.6: cannot open shared object file: No such file or directory /opt/VBoxGuestAdditions-7.0.6/bin/VBoxClient: error while loading shared libraries: libXt.so.6: cannot open shared object file: No such file or directory VirtualBox Guest Additions: Starting. VirtualBox Guest Additions: Setting up modules VirtualBox Guest Additions: Building the VirtualBox Guest Additions kernel modules. This may take a while. VirtualBox Guest Additions: To build modules for other installed kernels, run VirtualBox Guest Additions: /sbin/rcvboxadd quicksetup VirtualBox Guest Additions: or VirtualBox Guest Additions: /sbin/rcvboxadd quicksetup all VirtualBox Guest Additions: Building the modules for kernel 5.4.0-139-generic. update-initramfs: Generating /boot/initrd.img-5.4.0-139-generic Unmounting Virtualbox Guest Additions ISO from: /mnt ==> controlplane: Checking for guest additions in VM... ==> controlplane: Setting hostname... ==> controlplane: Configuring and enabling network interfaces... ==> controlplane: Mounting shared folders... controlplane: /var/sync/linux => F:/Kubernetes/sig-windows-dev-tools/sync/linux controlplane: /var/sync/forked => F:/Kubernetes/sig-windows-dev-tools/forked controlplane: /var/sync/shared => F:/Kubernetes/sig-windows-dev-tools/sync/shared ==> controlplane: Running provisioner: shell... controlplane: Running: C:/Users/MateuszL/AppData/Local/Temp/vagrant-shell20230416-8672-17xy4f5.sh controlplane: ARGS: 1.26 10.20.30.10 100.244.0.0/16 controlplane: Using 1.26 as the Kubernetes version controlplane: Setting up internet connectivity to /etc/resolv.conf controlplane: nameserver 8.8.8.8 controlplane: nameserver 1.1.1.1 controlplane: now curling to add keys... controlplane: Warning: apt-key output should not be parsed (stdout is not a terminal) controlplane: OK controlplane: deb https://apt.kubernetes.io/ kubernetes-xenial main controlplane: SWDT: Running apt get update -y controlplane: Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease controlplane: Hit:3 http://us.archive.ubuntu.com/ubuntu focal InRelease controlplane: Hit:4 http://us.archive.ubuntu.com/ubuntu focal-updates InRelease controlplane: Hit:5 http://us.archive.ubuntu.com/ubuntu focal-backports InRelease controlplane: Get:2 https://packages.cloud.google.com/apt kubernetes-xenial InRelease [8,993 B] controlplane: Get:6 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 Packages [65.7 kB] controlplane: Fetched 74.7 kB in 1s (66.4 kB/s) controlplane: Reading package lists... controlplane: overlay controlplane: br_netfilter controlplane: SWDT: Running modprobes controlplane: net.bridge.bridge-nf-call-iptables = 1 controlplane: net.ipv4.ip_forward = 1 controlplane: net.bridge.bridge-nf-call-ip6tables = 1 controlplane: * Applying /etc/sysctl.d/10-console-messages.conf ... controlplane: kernel.printk = 4 4 1 7 controlplane: * Applying /etc/sysctl.d/10-ipv6-privacy.conf ... controlplane: net.ipv6.conf.all.use_tempaddr = 2 controlplane: net.ipv6.conf.default.use_tempaddr = 2 controlplane: * Applying /etc/sysctl.d/10-kernel-hardening.conf ... controlplane: kernel.kptr_restrict = 1 controlplane: * Applying /etc/sysctl.d/10-link-restrictions.conf ... controlplane: fs.protected_hardlinks = 1 controlplane: fs.protected_symlinks = 1 controlplane: * Applying /etc/sysctl.d/10-magic-sysrq.conf ... controlplane: kernel.sysrq = 176 controlplane: * Applying /etc/sysctl.d/10-network-security.conf ... controlplane: net.ipv4.conf.default.rp_filter = 2 controlplane: net.ipv4.conf.all.rp_filter = 2 controlplane: * Applying /etc/sysctl.d/10-ptrace.conf ... controlplane: kernel.yama.ptrace_scope = 1 controlplane: * Applying /etc/sysctl.d/10-zeropage.conf ... controlplane: vm.mmap_min_addr = 65536 controlplane: * Applying /usr/lib/sysctl.d/50-default.conf ... controlplane: net.ipv4.conf.default.promote_secondaries = 1 controlplane: sysctl: setting key "net.ipv4.conf.all.promote_secondaries": Invalid argument controlplane: net.ipv4.ping_group_range = 0 2147483647 controlplane: net.core.default_qdisc = fq_codel controlplane: fs.protected_regular = 1 controlplane: fs.protected_fifos = 1 controlplane: * Applying /usr/lib/sysctl.d/50-pid-max.conf ... controlplane: kernel.pid_max = 4194304 controlplane: * Applying /etc/sysctl.d/99-kubernetes-cri.conf ... controlplane: net.bridge.bridge-nf-call-iptables = 1 controlplane: net.ipv4.ip_forward = 1 controlplane: net.bridge.bridge-nf-call-ip6tables = 1 controlplane: * Applying /etc/sysctl.d/99-sysctl.conf ... controlplane: net.ipv6.conf.all.disable_ipv6 = 1 controlplane: * Applying /usr/lib/sysctl.d/protect-links.conf ... controlplane: fs.protected_fifos = 1 controlplane: fs.protected_hardlinks = 1 controlplane: fs.protected_regular = 2 controlplane: fs.protected_symlinks = 1 controlplane: * Applying /etc/sysctl.conf ... controlplane: net.ipv6.conf.all.disable_ipv6 = 1 controlplane: SWDT installing kubelet, kubeadm, kubectl will overwrite them later as needeed... controlplane: Reading package lists... controlplane: Building dependency tree... controlplane: Reading state information... controlplane: The following additional packages will be installed: controlplane: conntrack cri-tools ebtables kubernetes-cni socat controlplane: Suggested packages: controlplane: nftables controlplane: The following NEW packages will be installed: controlplane: conntrack cri-tools ebtables kubeadm kubectl kubelet kubernetes-cni socat controlplane: 0 upgraded, 8 newly installed, 0 to remove and 70 not upgraded. controlplane: Need to get 85.9 MB of archives. controlplane: After this operation, 329 MB of additional disk space will be used. controlplane: Get:2 http://us.archive.ubuntu.com/ubuntu focal/main amd64 conntrack amd64 1:1.4.5-2 [30.3 kB] controlplane: Get:1 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 cri-tools amd64 1.26.0-00 [18.9 MB] controlplane: Get:5 http://us.archive.ubuntu.com/ubuntu focal/main amd64 ebtables amd64 2.0.11-3build1 [80.3 kB] controlplane: Get:8 http://us.archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] controlplane: Get:3 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 kubernetes-cni amd64 1.2.0-00 [27.6 MB] controlplane: Get:4 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 kubelet amd64 1.27.1-00 [18.7 MB] controlplane: Get:6 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 kubectl amd64 1.27.1-00 [10.2 MB] controlplane: Get:7 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 kubeadm amd64 1.27.1-00 [9,928 kB] controlplane: dpkg-preconfigure: unable to re-open stdin: No such file or directory controlplane: Fetched 85.9 MB in 10s (8,301 kB/s) controlplane: Selecting previously unselected package conntrack. (Reading database ... 117499 files and directories currently installed.) controlplane: Preparing to unpack .../0-conntrack_1%3a1.4.5-2_amd64.deb ... controlplane: Unpacking conntrack (1:1.4.5-2) ... controlplane: Selecting previously unselected package cri-tools. controlplane: Preparing to unpack .../1-cri-tools_1.26.0-00_amd64.deb ... controlplane: Unpacking cri-tools (1.26.0-00) ... controlplane: Selecting previously unselected package ebtables. controlplane: Preparing to unpack .../2-ebtables_2.0.11-3build1_amd64.deb ... controlplane: Unpacking ebtables (2.0.11-3build1) ... controlplane: Selecting previously unselected package kubernetes-cni. controlplane: Preparing to unpack .../3-kubernetes-cni_1.2.0-00_amd64.deb ... controlplane: Unpacking kubernetes-cni (1.2.0-00) ... controlplane: Selecting previously unselected package socat. controlplane: Preparing to unpack .../4-socat_1.7.3.3-2_amd64.deb ... controlplane: Unpacking socat (1.7.3.3-2) ... controlplane: Selecting previously unselected package kubelet. controlplane: Preparing to unpack .../5-kubelet_1.27.1-00_amd64.deb ... controlplane: Unpacking kubelet (1.27.1-00) ... controlplane: Selecting previously unselected package kubectl. controlplane: Preparing to unpack .../6-kubectl_1.27.1-00_amd64.deb ... controlplane: Unpacking kubectl (1.27.1-00) ... controlplane: Selecting previously unselected package kubeadm. controlplane: Preparing to unpack .../7-kubeadm_1.27.1-00_amd64.deb ... controlplane: Unpacking kubeadm (1.27.1-00) ... controlplane: Setting up conntrack (1:1.4.5-2) ... controlplane: Setting up kubectl (1.27.1-00) ... controlplane: Setting up ebtables (2.0.11-3build1) ... controlplane: Setting up socat (1.7.3.3-2) ... controlplane: Setting up cri-tools (1.26.0-00) ... controlplane: Setting up kubernetes-cni (1.2.0-00) ... controlplane: Setting up kubelet (1.27.1-00) ... controlplane: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service → /lib/systemd/system/kubelet.service. controlplane: Setting up kubeadm (1.27.1-00) ... controlplane: Processing triggers for man-db (2.9.1-1) ... controlplane: kubelet set on hold. controlplane: kubeadm set on hold. controlplane: kubectl set on hold. controlplane: Configuring Containerd controlplane: Reading package lists... controlplane: Building dependency tree... controlplane: Reading state information... controlplane: lsb-release is already the newest version (11.1.0ubuntu2). controlplane: ca-certificates is already the newest version (20211016ubuntu0.20.04.1). controlplane: gnupg is already the newest version (2.2.19-3ubuntu2.2). controlplane: 0 upgraded, 0 newly installed, 0 to remove and 70 not upgraded. controlplane: Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease controlplane: Get:2 https://download.docker.com/linux/ubuntu focal InRelease [57.7 kB] controlplane: Hit:3 http://us.archive.ubuntu.com/ubuntu focal InRelease controlplane: Hit:5 http://us.archive.ubuntu.com/ubuntu focal-updates InRelease controlplane: Hit:4 https://packages.cloud.google.com/apt kubernetes-xenial InRelease controlplane: Hit:6 http://us.archive.ubuntu.com/ubuntu focal-backports InRelease controlplane: Get:7 https://download.docker.com/linux/ubuntu focal/stable amd64 Packages [26.8 kB] controlplane: Fetched 84.5 kB in 1s (75.1 kB/s) controlplane: Reading package lists... controlplane: Reading package lists... controlplane: Building dependency tree... controlplane: Reading state information... controlplane: The following NEW packages will be installed: controlplane: containerd.io controlplane: 0 upgraded, 1 newly installed, 0 to remove and 70 not upgraded. controlplane: Need to get 28.3 MB of archives. controlplane: After this operation, 116 MB of additional disk space will be used. controlplane: Get:1 https://download.docker.com/linux/ubuntu focal/stable amd64 containerd.io amd64 1.6.20-1 [28.3 MB] controlplane: dpkg-preconfigure: unable to re-open stdin: No such file or directory controlplane: Fetched 28.3 MB in 3s (8,287 kB/s) controlplane: Selecting previously unselected package containerd.io. (Reading database ... 117593 files and directories currently installed.) controlplane: Preparing to unpack .../containerd.io_1.6.20-1_amd64.deb ... controlplane: Unpacking containerd.io (1.6.20-1) ... controlplane: Setting up containerd.io (1.6.20-1) ... controlplane: Created symlink /etc/systemd/system/multi-user.target.wants/containerd.service → /lib/systemd/system/containerd.service. controlplane: Processing triggers for man-db (2.9.1-1) ... controlplane: copying /var/sync/linux/bin/kubeadm to node path.. controlplane: copying /var/sync/linux/bin/kubectl to node path.. controlplane: copying /var/sync/linux/bin/kubelet to node path.. controlplane: disabled_plugins = [] controlplane: imports = [] controlplane: oom_score = 0 controlplane: plugin_dir = "" controlplane: required_plugins = [] controlplane: root = "/var/lib/containerd" controlplane: state = "/run/containerd" controlplane: temp = "" controlplane: version = 2 controlplane: controlplane: [cgroup] controlplane: path = "" controlplane: controlplane: [debug] controlplane: address = "" controlplane: format = "" controlplane: gid = 0 controlplane: level = "" controlplane: uid = 0 controlplane: controlplane: [grpc] controlplane: address = "/run/containerd/containerd.sock" controlplane: gid = 0 controlplane: max_recv_message_size = 16777216 controlplane: max_send_message_size = 16777216 controlplane: tcp_address = "" controlplane: tcp_tls_ca = "" controlplane: tcp_tls_cert = "" controlplane: tcp_tls_key = "" controlplane: uid = 0 controlplane: controlplane: [metrics] controlplane: address = "" controlplane: grpc_histogram = false controlplane: controlplane: [plugins] controlplane: controlplane: [plugins."io.containerd.gc.v1.scheduler"] controlplane: deletion_threshold = 0 controlplane: mutation_threshold = 100 controlplane: pause_threshold = 0.02 controlplane: schedule_delay = "0s" controlplane: startup_delay = "100ms" controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri"] controlplane: device_ownership_from_security_context = false controlplane: disable_apparmor = false controlplane: disable_cgroup = false controlplane: disable_hugetlb_controller = true controlplane: disable_proc_mount = false controlplane: disable_tcp_service = true controlplane: enable_selinux = false controlplane: enable_tls_streaming = false controlplane: enable_unprivileged_icmp = false controlplane: enable_unprivileged_ports = false controlplane: ignore_image_defined_volumes = false controlplane: max_concurrent_downloads = 3 controlplane: max_container_log_line_size = 16384 controlplane: netns_mounts_under_state_dir = false controlplane: restrict_oom_score_adj = false controlplane: sandbox_image = "registry.k8s.io/pause:3.6" controlplane: selinux_category_range = 1024 controlplane: stats_collect_period = 10 controlplane: stream_idle_timeout = "4h0m0s" controlplane: stream_server_address = "127.0.0.1" controlplane: stream_server_port = "0" controlplane: systemd_cgroup = false controlplane: tolerate_missing_hugetlb_controller = true controlplane: unset_seccomp_profile = "" controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".cni] controlplane: bin_dir = "/opt/cni/bin" controlplane: conf_dir = "/etc/cni/net.d" controlplane: conf_template = "" controlplane: ip_pref = "" controlplane: max_conf_num = 1 controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".containerd] controlplane: default_runtime_name = "runc" controlplane: disable_snapshot_annotations = true controlplane: discard_unpacked_layers = false controlplane: ignore_rdt_not_enabled_errors = false controlplane: no_pivot = false controlplane: snapshotter = "overlayfs" controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".containerd.default_runtime] controlplane: base_runtime_spec = "" controlplane: cni_conf_dir = "" controlplane: cni_max_conf_num = 0 controlplane: container_annotations = [] controlplane: pod_annotations = [] controlplane: privileged_without_host_devices = false controlplane: runtime_engine = "" controlplane: runtime_path = "" controlplane: runtime_root = "" controlplane: runtime_type = "" controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".containerd.default_runtime.options] controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".containerd.runtimes] controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc] controlplane: base_runtime_spec = "" controlplane: cni_conf_dir = "" controlplane: cni_max_conf_num = 0 controlplane: container_annotations = [] controlplane: pod_annotations = [] controlplane: privileged_without_host_devices = false controlplane: runtime_engine = "" controlplane: runtime_path = "" controlplane: runtime_root = "" controlplane: runtime_type = "io.containerd.runc.v2" controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options] controlplane: BinaryName = "" controlplane: CriuImagePath = "" controlplane: CriuPath = "" controlplane: CriuWorkPath = "" controlplane: IoGid = 0 controlplane: IoUid = 0 controlplane: NoNewKeyring = false controlplane: NoPivotRoot = false controlplane: Root = "" controlplane: ShimCgroup = "" controlplane: SystemdCgroup = false controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".containerd.untrusted_workload_runtime] controlplane: base_runtime_spec = "" controlplane: cni_conf_dir = "" controlplane: cni_max_conf_num = 0 controlplane: container_annotations = [] controlplane: pod_annotations = [] controlplane: privileged_without_host_devices = false controlplane: runtime_engine = "" controlplane: runtime_path = "" controlplane: runtime_root = "" controlplane: runtime_type = "" controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".containerd.untrusted_workload_runtime.options] controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".image_decryption] controlplane: key_model = "node" controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".registry] controlplane: config_path = "" controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".registry.auths] controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".registry.configs] controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".registry.headers] controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".registry.mirrors] controlplane: controlplane: [plugins."io.containerd.grpc.v1.cri".x509_key_pair_streaming] controlplane: tls_cert_file = "" controlplane: tls_key_file = "" controlplane: controlplane: [plugins."io.containerd.internal.v1.opt"] controlplane: path = "/opt/containerd" controlplane: controlplane: [plugins."io.containerd.internal.v1.restart"] controlplane: interval = "10s" controlplane: controlplane: [plugins."io.containerd.internal.v1.tracing"] controlplane: sampling_ratio = 1.0 controlplane: service_name = "containerd" controlplane: controlplane: [plugins."io.containerd.metadata.v1.bolt"] controlplane: content_sharing_policy = "shared" controlplane: controlplane: [plugins."io.containerd.monitor.v1.cgroups"] controlplane: no_prometheus = false controlplane: controlplane: [plugins."io.containerd.runtime.v1.linux"] controlplane: no_shim = false controlplane: runtime = "runc" controlplane: runtime_root = "" controlplane: shim = "containerd-shim" controlplane: shim_debug = false controlplane: controlplane: [plugins."io.containerd.runtime.v2.task"] controlplane: platforms = ["linux/amd64"] controlplane: sched_core = false controlplane: controlplane: [plugins."io.containerd.service.v1.diff-service"] controlplane: default = ["walking"] controlplane: controlplane: [plugins."io.containerd.service.v1.tasks-service"] controlplane: rdt_config_file = "" controlplane: controlplane: [plugins."io.containerd.snapshotter.v1.aufs"] controlplane: root_path = "" controlplane: controlplane: [plugins."io.containerd.snapshotter.v1.btrfs"] controlplane: root_path = "" controlplane: controlplane: [plugins."io.containerd.snapshotter.v1.devmapper"] controlplane: async_remove = false controlplane: base_image_size = "" controlplane: discard_blocks = false controlplane: fs_options = "" controlplane: fs_type = "" controlplane: pool_name = "" controlplane: root_path = "" controlplane: controlplane: [plugins."io.containerd.snapshotter.v1.native"] controlplane: root_path = "" controlplane: controlplane: [plugins."io.containerd.snapshotter.v1.overlayfs"] controlplane: root_path = "" controlplane: upperdir_label = false controlplane: controlplane: [plugins."io.containerd.snapshotter.v1.zfs"] controlplane: root_path = "" controlplane: controlplane: [plugins."io.containerd.tracing.processor.v1.otlp"] controlplane: endpoint = "" controlplane: insecure = false controlplane: protocol = "" controlplane: controlplane: [proxy_plugins] controlplane: controlplane: [stream_processors] controlplane: controlplane: [stream_processors."io.containerd.ocicrypt.decoder.v1.tar"] controlplane: accepts = ["application/vnd.oci.image.layer.v1.tar+encrypted"] controlplane: args = ["--decryption-keys-path", "/etc/containerd/ocicrypt/keys"] controlplane: env = ["OCICRYPT_KEYPROVIDER_CONFIG=/etc/containerd/ocicrypt/ocicrypt_keyprovider.conf"] controlplane: path = "ctd-decoder" controlplane: returns = "application/vnd.oci.image.layer.v1.tar" controlplane: controlplane: [stream_processors."io.containerd.ocicrypt.decoder.v1.tar.gzip"] controlplane: accepts = ["application/vnd.oci.image.layer.v1.tar+gzip+encrypted"] controlplane: args = ["--decryption-keys-path", "/etc/containerd/ocicrypt/keys"] controlplane: env = ["OCICRYPT_KEYPROVIDER_CONFIG=/etc/containerd/ocicrypt/ocicrypt_keyprovider.conf"] controlplane: path = "ctd-decoder" controlplane: returns = "application/vnd.oci.image.layer.v1.tar+gzip" controlplane: controlplane: [timeouts] controlplane: "io.containerd.timeout.bolt.open" = "0s" controlplane: "io.containerd.timeout.shim.cleanup" = "5s" controlplane: "io.containerd.timeout.shim.load" = "5s" controlplane: "io.containerd.timeout.shim.shutdown" = "3s" controlplane: "io.containerd.timeout.task.state" = "2s" controlplane: controlplane: [ttrpc] controlplane: address = "" controlplane: gid = 0 controlplane: uid = 0 controlplane: I0416 10:25:51.046739 38980 initconfiguration.go:254] loading configuration from "/var/sync/shared/kubeadm.yaml" controlplane: I0416 10:25:51.053209 38980 initconfiguration.go:116] detected and using CRI socket: unix:///var/run/containerd/containerd.sock controlplane: I0416 10:25:51.053637 38980 kubelet.go:196] the value of KubeletConfiguration.cgroupDriver is empty; setting it to "systemd" controlplane: I0416 10:25:51.064925 38980 version.go:187] fetching Kubernetes version from URL: https://dl.k8s.io/release/stable-1.26.txt controlplane: I0416 10:25:51.504639 38980 common.go:128] WARNING: tolerating control plane version v1.26.4 as a pre-release version controlplane: [init] Using Kubernetes version: v1.26.4 controlplane: [preflight] Running pre-flight checks controlplane: I0416 10:25:51.509192 38980 checks.go:568] validating Kubernetes and kubeadm version controlplane: I0416 10:25:51.509729 38980 checks.go:168] validating if the firewall is enabled and active controlplane: I0416 10:25:51.528739 38980 checks.go:203] validating availability of port 6443 controlplane: I0416 10:25:51.531553 38980 checks.go:203] validating availability of port 10259 controlplane: I0416 10:25:51.531736 38980 checks.go:203] validating availability of port 10257 controlplane: I0416 10:25:51.531776 38980 checks.go:280] validating the existence of file /etc/kubernetes/manifests/kube-apiserver.yaml controlplane: I0416 10:25:51.531802 38980 checks.go:280] validating the existence of file /etc/kubernetes/manifests/kube-controller-manager.yaml controlplane: I0416 10:25:51.531814 38980 checks.go:280] validating the existence of file /etc/kubernetes/manifests/kube-scheduler.yaml controlplane: I0416 10:25:51.531824 38980 checks.go:280] validating the existence of file /etc/kubernetes/manifests/etcd.yaml controlplane: I0416 10:25:51.531844 38980 checks.go:430] validating if the connectivity type is via proxy or direct controlplane: I0416 10:25:51.531874 38980 checks.go:469] validating http connectivity to first IP address in the CIDR controlplane: I0416 10:25:51.531899 38980 checks.go:469] validating http connectivity to first IP address in the CIDR controlplane: I0416 10:25:51.531917 38980 checks.go:104] validating the container runtime controlplane: I0416 10:25:51.599565 38980 checks.go:329] validating the contents of file /proc/sys/net/bridge/bridge-nf-call-iptables controlplane: I0416 10:25:51.600084 38980 checks.go:329] validating the contents of file /proc/sys/net/ipv4/ip_forward controlplane: I0416 10:25:51.600119 38980 checks.go:644] validating whether swap is enabled or not controlplane: I0416 10:25:51.600158 38980 checks.go:370] validating the presence of executable crictl controlplane: I0416 10:25:51.600180 38980 checks.go:370] validating the presence of executable conntrack controlplane: I0416 10:25:51.600212 38980 checks.go:370] validating the presence of executable ip controlplane: I0416 10:25:51.600233 38980 checks.go:370] validating the presence of executable iptables controlplane: I0416 10:25:51.600284 38980 checks.go:370] validating the presence of executable mount controlplane: I0416 10:25:51.600299 38980 checks.go:370] validating the presence of executable nsenter controlplane: I0416 10:25:51.600327 38980 checks.go:370] validating the presence of executable ebtables controlplane: I0416 10:25:51.600351 38980 checks.go:370] validating the presence of executable ethtool controlplane: I0416 10:25:51.600367 38980 checks.go:370] validating the presence of executable socat controlplane: I0416 10:25:51.600384 38980 checks.go:370] validating the presence of executable tc controlplane: I0416 10:25:51.600406 38980 checks.go:370] validating the presence of executable touch controlplane: I0416 10:25:51.600419 38980 checks.go:516] running all checks controlplane: I0416 10:25:51.637534 38980 checks.go:401] checking whether the given node name is valid and reachable using net.LookupHost controlplane: I0416 10:25:51.637742 38980 checks.go:610] validating kubelet version controlplane: I0416 10:25:51.775083 38980 checks.go:130] validating if the "kubelet" service is enabled and active controlplane: [preflight] Pulling images required for setting up a Kubernetes cluster controlplane: [preflight] This might take a minute or two, depending on the speed of your internet connection controlplane: [preflight] You can also perform this action in beforehand using 'kubeadm config images pull' controlplane: I0416 10:25:51.789653 38980 checks.go:203] validating availability of port 10250 controlplane: I0416 10:25:51.789723 38980 checks.go:203] validating availability of port 2379 controlplane: I0416 10:25:51.789747 38980 checks.go:203] validating availability of port 2380 controlplane: I0416 10:25:51.789769 38980 checks.go:243] validating the existence and emptiness of directory /var/lib/etcd controlplane: I0416 10:25:51.789854 38980 checks.go:832] using image pull policy: IfNotPresent controlplane: I0416 10:25:51.864237 38980 checks.go:849] pulling: registry.k8s.io/kube-apiserver:v1.26.4 controlplane: I0416 10:26:01.497908 38980 checks.go:849] pulling: registry.k8s.io/kube-controller-manager:v1.26.4 controlplane: I0416 10:26:09.760468 38980 checks.go:849] pulling: registry.k8s.io/kube-scheduler:v1.26.4 controlplane: I0416 10:26:14.422180 38980 checks.go:849] pulling: registry.k8s.io/kube-proxy:v1.26.4 controlplane: I0416 10:26:22.688487 38980 checks.go:849] pulling: registry.k8s.io/pause:3.9 controlplane: I0416 10:26:24.272234 38980 checks.go:849] pulling: registry.k8s.io/etcd:3.5.6-0 controlplane: I0416 10:26:46.889947 38980 checks.go:849] pulling: registry.k8s.io/coredns/coredns:v1.9.3 controlplane: [certs] Using certificateDir folder "/etc/kubernetes/pki" controlplane: I0416 10:26:52.625726 38980 certs.go:112] creating a new certificate authority for ca controlplane: [certs] Generating "ca" certificate and key controlplane: I0416 10:26:53.083506 38980 certs.go:519] validating certificate period for ca certificate controlplane: [certs] Generating "apiserver" certificate and key controlplane: [certs] apiserver serving cert is signed for DNS names [controlplane kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 10.20.30.10] controlplane: [certs] Generating "apiserver-kubelet-client" certificate and key controlplane: I0416 10:26:53.673048 38980 certs.go:112] creating a new certificate authority for front-proxy-ca controlplane: [certs] Generating "front-proxy-ca" certificate and key controlplane: I0416 10:26:53.933117 38980 certs.go:519] validating certificate period for front-proxy-ca certificate controlplane: [certs] Generating "front-proxy-client" certificate and key controlplane: I0416 10:26:54.115335 38980 certs.go:112] creating a new certificate authority for etcd-ca controlplane: [certs] Generating "etcd/ca" certificate and key controlplane: I0416 10:26:54.387972 38980 certs.go:519] validating certificate period for etcd/ca certificate controlplane: [certs] Generating "etcd/server" certificate and key controlplane: [certs] etcd/server serving cert is signed for DNS names [controlplane localhost] and IPs [10.20.30.10 127.0.0.1 ::1] controlplane: [certs] Generating "etcd/peer" certificate and key controlplane: [certs] etcd/peer serving cert is signed for DNS names [controlplane localhost] and IPs [10.20.30.10 127.0.0.1 ::1] controlplane: [certs] Generating "etcd/healthcheck-client" certificate and key controlplane: [certs] Generating "apiserver-etcd-client" certificate and key controlplane: I0416 10:26:55.177615 38980 certs.go:78] creating new public/private key files for signing service account users controlplane: [certs] Generating "sa" key and public key controlplane: [kubeconfig] Using kubeconfig folder "/etc/kubernetes" controlplane: I0416 10:26:55.648398 38980 kubeconfig.go:103] creating kubeconfig file for admin.conf controlplane: [kubeconfig] Writing "admin.conf" kubeconfig file controlplane: I0416 10:26:56.028574 38980 kubeconfig.go:103] creating kubeconfig file for kubelet.conf controlplane: [kubeconfig] Writing "kubelet.conf" kubeconfig file controlplane: I0416 10:26:56.560685 38980 kubeconfig.go:103] creating kubeconfig file for controller-manager.conf controlplane: [kubeconfig] Writing "controller-manager.conf" kubeconfig file controlplane: I0416 10:26:57.524903 38980 kubeconfig.go:103] creating kubeconfig file for scheduler.conf controlplane: [kubeconfig] Writing "scheduler.conf" kubeconfig file controlplane: I0416 10:26:57.642756 38980 kubelet.go:67] Stopping the kubelet controlplane: [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" controlplane: [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" controlplane: [kubelet-start] Starting the kubelet controlplane: [control-plane] Using manifest folder "/etc/kubernetes/manifests" controlplane: [control-plane] Creating static Pod manifest for "kube-apiserver" controlplane: [control-plane] Creating static Pod manifest for "kube-controller-manager" controlplane: [control-plane] Creating static Pod manifest for "kube-scheduler" controlplane: [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests" controlplane: I0416 10:26:58.023434 38980 manifests.go:99] [control-plane] getting StaticPodSpecs controlplane: I0416 10:26:58.023700 38980 certs.go:519] validating certificate period for CA certificate controlplane: I0416 10:26:58.023803 38980 manifests.go:125] [control-plane] adding volume "ca-certs" for component "kube-apiserver" controlplane: I0416 10:26:58.023811 38980 manifests.go:125] [control-plane] adding volume "etc-ca-certificates" for component "kube-apiserver" controlplane: I0416 10:26:58.023816 38980 manifests.go:125] [control-plane] adding volume "etc-pki" for component "kube-apiserver" controlplane: I0416 10:26:58.023822 38980 manifests.go:125] [control-plane] adding volume "k8s-certs" for component "kube-apiserver" controlplane: I0416 10:26:58.023827 38980 manifests.go:125] [control-plane] adding volume "usr-local-share-ca-certificates" for component "kube-apiserver" controlplane: I0416 10:26:58.023832 38980 manifests.go:125] [control-plane] adding volume "usr-share-ca-certificates" for component "kube-apiserver" controlplane: I0416 10:26:58.026944 38980 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-apiserver" to "/etc/kubernetes/manifests/kube-apiserver.yaml" controlplane: I0416 10:26:58.026957 38980 manifests.go:99] [control-plane] getting StaticPodSpecs controlplane: I0416 10:26:58.027156 38980 manifests.go:125] [control-plane] adding volume "ca-certs" for component "kube-controller-manager" controlplane: I0416 10:26:58.027164 38980 manifests.go:125] [control-plane] adding volume "etc-ca-certificates" for component "kube-controller-manager" controlplane: I0416 10:26:58.027169 38980 manifests.go:125] [control-plane] adding volume "etc-pki" for component "kube-controller-manager" controlplane: I0416 10:26:58.027174 38980 manifests.go:125] [control-plane] adding volume "flexvolume-dir" for component "kube-controller-manager" controlplane: I0416 10:26:58.027179 38980 manifests.go:125] [control-plane] adding volume "k8s-certs" for component "kube-controller-manager" controlplane: I0416 10:26:58.027184 38980 manifests.go:125] [control-plane] adding volume "kubeconfig" for component "kube-controller-manager" controlplane: I0416 10:26:58.027189 38980 manifests.go:125] [control-plane] adding volume "usr-local-share-ca-certificates" for component "kube-controller-manager" controlplane: I0416 10:26:58.027205 38980 manifests.go:125] [control-plane] adding volume "usr-share-ca-certificates" for component "kube-controller-manager" controlplane: I0416 10:26:58.028004 38980 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-controller-manager" to "/etc/kubernetes/manifests/kube-controller-manager.yaml" controlplane: I0416 10:26:58.028016 38980 manifests.go:99] [control-plane] getting StaticPodSpecs controlplane: I0416 10:26:58.028219 38980 manifests.go:125] [control-plane] adding volume "kubeconfig" for component "kube-scheduler" controlplane: I0416 10:26:58.028709 38980 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-scheduler" to "/etc/kubernetes/manifests/kube-scheduler.yaml" controlplane: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s controlplane: I0416 10:26:58.049272 38980 local.go:65] [etcd] wrote Static Pod manifest for a local etcd member to "/etc/kubernetes/manifests/etcd.yaml" controlplane: I0416 10:26:58.049292 38980 waitcontrolplane.go:83] [wait-control-plane] Waiting for the API server to be healthy controlplane: I0416 10:26:58.049967 38980 loader.go:373] Config loaded from file: /etc/kubernetes/admin.conf controlplane: I0416 10:26:58.059768 38980 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s in 9 milliseconds controlplane: I0416 10:26:58.562006 38980 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s in 0 milliseconds controlplane: I0416 10:26:59.063718 38980 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s in 0 milliseconds controlplane: I0416 10:26:59.562370 38980 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s in 0 milliseconds controlplane: I0416 10:27:00.067500 38980 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s in 0 milliseconds controlplane: I0416 10:27:00.561154 38980 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s in 0 milliseconds controlplane: I0416 10:27:01.062533 38980 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s in 0 milliseconds controlplane: I0416 10:27:01.561302 38980 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s in 0 milliseconds controlplane: I0416 10:27:02.063023 38980 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s in 0 milliseconds controlplane: I0416 10:27:02.561076 38980 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s in 0 milliseconds controlplane: I0416 10:27:08.079344 38980 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s 500 Internal Server Error in 5012 milliseconds controlplane: I0416 10:27:08.563482 38980 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s 500 Internal Server Error in 2 milliseconds controlplane: I0416 10:27:09.066243 38980 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s 500 Internal Server Error in 5 milliseconds controlplane: I0416 10:27:09.562863 38980 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s 500 Internal Server Error in 2 milliseconds controlplane: I0416 10:27:10.067331 38980 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s 500 Internal Server Error in 6 milliseconds controlplane: [apiclient] All control plane components are healthy after 12.511540 seconds controlplane: [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace controlplane: I0416 10:27:10.562042 38980 round_trippers.go:553] GET https://10.20.30.10:6443/healthz?timeout=10s 200 OK in 1 milliseconds controlplane: I0416 10:27:10.562111 38980 uploadconfig.go:111] [upload-config] Uploading the kubeadm ClusterConfiguration to a ConfigMap controlplane: I0416 10:27:10.573108 38980 round_trippers.go:553] POST https://10.20.30.10:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s 201 Created in 9 milliseconds controlplane: I0416 10:27:10.580063 38980 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s 201 Created in 5 milliseconds controlplane: I0416 10:27:10.590341 38980 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s 201 Created in 7 milliseconds controlplane: I0416 10:27:10.591748 38980 uploadconfig.go:125] [upload-config] Uploading the kubelet component config to a ConfigMap controlplane: [kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster controlplane: I0416 10:27:10.604891 38980 round_trippers.go:553] POST https://10.20.30.10:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s 201 Created in 11 milliseconds controlplane: I0416 10:27:10.615739 38980 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s 201 Created in 10 milliseconds controlplane: I0416 10:27:10.624560 38980 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s 201 Created in 6 milliseconds controlplane: I0416 10:27:10.624801 38980 uploadconfig.go:130] [upload-config] Preserving the CRISocket information for the control-plane node controlplane: I0416 10:27:10.624908 38980 patchnode.go:31] [patchnode] Uploading the CRI Socket information "unix:///var/run/containerd/containerd.sock" to the Node API object "controlplane" as an annotation controlplane: I0416 10:27:11.131098 38980 round_trippers.go:553] GET https://10.20.30.10:6443/api/v1/nodes/controlplane?timeout=10s 200 OK in 3 milliseconds controlplane: I0416 10:27:11.145539 38980 round_trippers.go:553] PATCH https://10.20.30.10:6443/api/v1/nodes/controlplane?timeout=10s 200 OK in 12 milliseconds controlplane: [upload-certs] Skipping phase. Please see --upload-certs controlplane: [mark-control-plane] Marking the node controlplane as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers] controlplane: [mark-control-plane] Marking the node controlplane as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule] controlplane: I0416 10:27:11.651782 38980 round_trippers.go:553] GET https://10.20.30.10:6443/api/v1/nodes/controlplane?timeout=10s 200 OK in 3 milliseconds controlplane: I0416 10:27:11.680597 38980 round_trippers.go:553] PATCH https://10.20.30.10:6443/api/v1/nodes/controlplane?timeout=10s 200 OK in 26 milliseconds controlplane: [bootstrap-token] Using token: muvmwi.4w8v61l6dm13tpea controlplane: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles controlplane: I0416 10:27:11.690153 38980 round_trippers.go:553] GET https://10.20.30.10:6443/api/v1/namespaces/kube-system/secrets/bootstrap-token-muvmwi?timeout=10s 404 Not Found in 5 milliseconds controlplane: I0416 10:27:11.705560 38980 round_trippers.go:553] POST https://10.20.30.10:6443/api/v1/namespaces/kube-system/secrets?timeout=10s 201 Created in 13 milliseconds controlplane: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes controlplane: I0416 10:27:11.714605 38980 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/clusterroles?timeout=10s 201 Created in 7 milliseconds controlplane: I0416 10:27:11.729166 38980 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 12 milliseconds controlplane: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials controlplane: I0416 10:27:11.752330 38980 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 21 milliseconds controlplane: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token controlplane: I0416 10:27:11.773189 38980 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 19 milliseconds controlplane: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster controlplane: I0416 10:27:11.780755 38980 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 6 milliseconds controlplane: [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace controlplane: I0416 10:27:11.783089 38980 clusterinfo.go:47] [bootstrap-token] loading admin kubeconfig controlplane: I0416 10:27:11.784814 38980 loader.go:373] Config loaded from file: /etc/kubernetes/admin.conf controlplane: I0416 10:27:11.784966 38980 clusterinfo.go:58] [bootstrap-token] copying the cluster from admin.conf to the bootstrap kubeconfig controlplane: I0416 10:27:11.785212 38980 clusterinfo.go:70] [bootstrap-token] creating/updating ConfigMap in kube-public namespace controlplane: I0416 10:27:11.801130 38980 round_trippers.go:553] POST https://10.20.30.10:6443/api/v1/namespaces/kube-public/configmaps?timeout=10s 201 Created in 15 milliseconds controlplane: I0416 10:27:11.801900 38980 clusterinfo.go:84] creating the RBAC rules for exposing the cluster-info ConfigMap in the kube-public namespace controlplane: I0416 10:27:11.809474 38980 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-public/roles?timeout=10s 201 Created in 7 milliseconds controlplane: I0416 10:27:11.821813 38980 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-public/rolebindings?timeout=10s 201 Created in 9 milliseconds controlplane: I0416 10:27:11.822643 38980 kubeletfinalize.go:90] [kubelet-finalize] Assuming that kubelet client certificate rotation is enabled: found "/var/lib/kubelet/pki/kubelet-client-current.pem" controlplane: [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key controlplane: I0416 10:27:11.823280 38980 loader.go:373] Config loaded from file: /etc/kubernetes/kubelet.conf controlplane: I0416 10:27:11.824383 38980 kubeletfinalize.go:134] [kubelet-finalize] Restarting the kubelet to enable client certificate rotation controlplane: I0416 10:27:12.481823 38980 round_trippers.go:553] GET https://10.20.30.10:6443/apis/apps/v1/namespaces/kube-system/deployments?labelSelector=k8s-app%3Dkube-dns 200 OK in 7 milliseconds controlplane: I0416 10:27:12.520684 38980 round_trippers.go:553] GET https://10.20.30.10:6443/api/v1/namespaces/kube-system/configmaps/coredns?timeout=10s 404 Not Found in 4 milliseconds controlplane: I0416 10:27:12.530077 38980 round_trippers.go:553] POST https://10.20.30.10:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s 201 Created in 9 milliseconds controlplane: I0416 10:27:12.539722 38980 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/clusterroles?timeout=10s 201 Created in 9 milliseconds controlplane: I0416 10:27:12.547630 38980 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 6 milliseconds controlplane: I0416 10:27:12.555975 38980 round_trippers.go:553] POST https://10.20.30.10:6443/api/v1/namespaces/kube-system/serviceaccounts?timeout=10s 201 Created in 7 milliseconds controlplane: I0416 10:27:12.585987 38980 round_trippers.go:553] POST https://10.20.30.10:6443/apis/apps/v1/namespaces/kube-system/deployments?timeout=10s 201 Created in 27 milliseconds controlplane: I0416 10:27:12.606939 38980 round_trippers.go:553] POST https://10.20.30.10:6443/api/v1/namespaces/kube-system/services?timeout=10s 201 Created in 17 milliseconds controlplane: [addons] Applied essential addon: CoreDNS controlplane: I0416 10:27:12.624340 38980 round_trippers.go:553] POST https://10.20.30.10:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s 201 Created in 15 milliseconds controlplane: I0416 10:27:12.649717 38980 round_trippers.go:553] POST https://10.20.30.10:6443/apis/apps/v1/namespaces/kube-system/daemonsets?timeout=10s 201 Created in 20 milliseconds controlplane: I0416 10:27:12.669768 38980 round_trippers.go:553] POST https://10.20.30.10:6443/api/v1/namespaces/kube-system/serviceaccounts?timeout=10s 201 Created in 19 milliseconds controlplane: I0416 10:27:12.683591 38980 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 13 milliseconds controlplane: I0416 10:27:12.702832 38980 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s 201 Created in 17 milliseconds controlplane: [addons] Applied essential addon: kube-proxy controlplane: I0416 10:27:12.711952 38980 round_trippers.go:553] POST https://10.20.30.10:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s 201 Created in 8 milliseconds controlplane: I0416 10:27:12.713951 38980 loader.go:373] Config loaded from file: /etc/kubernetes/admin.conf controlplane: controlplane: Your Kubernetes control-plane has initialized successfully! controlplane: controlplane: To start using your cluster, you need to run the following as a regular user: controlplane: controlplane: mkdir -p $HOME/.kube controlplane: sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config controlplane: sudo chown $(id -u):$(id -g) $HOME/.kube/config controlplane: controlplane: Alternatively, if you are the root user, you can run: controlplane: controlplane: export KUBECONFIG=/etc/kubernetes/admin.conf controlplane: controlplane: You should now deploy a pod network to the cluster. controlplane: Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: controlplane: https://kubernetes.io/docs/concepts/cluster-administration/addons/ controlplane: controlplane: Then you can join any number of worker nodes by running the following on each as root: controlplane: controlplane: kubeadm join 10.20.30.10:6443 --token muvmwi.4w8v61l6dm13tpea \ controlplane: --discovery-token-ca-cert-hash sha256:0eadc1b37c002386f79ebb438a4ceb77215dd6487a1c000aa85f074648a02d4e controlplane: I0416 10:27:12.717765 38980 loader.go:373] Config loaded from file: /etc/kubernetes/admin.conf controlplane: serviceaccount/kube-proxy-windows created controlplane: clusterrolebinding.rbac.authorization.k8s.io/node:kube-proxy created controlplane: clusterrolebinding.rbac.authorization.k8s.io/node:god2 created controlplane: clusterrolebinding.rbac.authorization.k8s.io/node:god3 created controlplane: clusterrolebinding.rbac.authorization.k8s.io/node:god4 created controlplane: Testing controlplane nodes! controlplane: NAMESPACE NAME READY STATUS RESTARTS AGE controlplane: kube-system etcd-controlplane 0/1 Pending 0 1s controlplane: kube-system kube-apiserver-controlplane 1/1 Running 0 1s controlplane: kube-system kube-controller-manager-controlplane 1/1 Running 0 1s controlplane: kube-system kube-scheduler-controlplane 1/1 Running 0 3s ==> controlplane: Running provisioner: shell... controlplane: Running: C:/Users/MateuszL/AppData/Local/Temp/vagrant-shell20230416-8672-1mn2dvs.sh controlplane: running calico installer now with pod_cidr 100.244.0.0/16 controlplane: node/controlplane untainted controlplane: error: taint "node-role.kubernetes.io/master" not found controlplane: namespace/calico-system created controlplane: namespace/tigera-operator created controlplane: customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created controlplane: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io created controlplane: customresourcedefinition.apiextensions.k8s.io/imagesets.operator.tigera.io created controlplane: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io created controlplane: customresourcedefinition.apiextensions.k8s.io/tigerastatuses.operator.tigera.io created controlplane: serviceaccount/tigera-operator created controlplane: clusterrole.rbac.authorization.k8s.io/tigera-operator created controlplane: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created controlplane: deployment.apps/tigera-operator created controlplane: --2023-04-16 10:27:21-- https://raw.githubusercontent.com/projectcalico/calico/v3.25.0/manifests/custom-resources.yaml controlplane: Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.110.133, 185.199.111.133, 185.199.109.133, ... controlplane: Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.110.133|:443... connected. controlplane: HTTP request sent, awaiting response... 200 OK controlplane: Length: 827 [text/plain] controlplane: Saving to: ‘trigera-custom-resource.yaml’ controlplane: controlplane: 0K 100% 38.0M=0s controlplane: controlplane: 2023-04-16 10:27:22 (38.0 MB/s) - ‘trigera-custom-resource.yaml’ saved [827/827] controlplane: controlplane: installation.operator.tigera.io/default created controlplane: apiserver.operator.tigera.io/default created controlplane: installation.operator.tigera.io/default patched controlplane: waiting 20s for calico pods... controlplane: --2023-04-16 10:27:47-- https://raw.githubusercontent.com/projectcalico/calico/v3.25.0/manifests/calico-windows-vxlan.yaml controlplane: Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ... controlplane: Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected. controlplane: HTTP request sent, awaiting response... 200 OK controlplane: Length: 4157 (4.1K) [text/plain] controlplane: Saving to: ‘calico-windows.yaml’ controlplane: controlplane: 0K .... 100% 14.9M=0s controlplane: controlplane: 2023-04-16 10:27:47 (14.9 MB/s) - ‘calico-windows.yaml’ saved [4157/4157] controlplane: controlplane: configmap/calico-windows-config created controlplane: daemonset.apps/calico-node-windows created controlplane: % Total % Received % Xferd Average Speed Time Time Time Current controlplane: Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 56.8M 100 56.8M 0 0 6511k 0 0:00:08 0:00:08 --:--:-- 7096k controlplane: Successfully set StrictAffinity to: true controlplane: NAME READY STATUS RESTARTS AGE controlplane: calico-kube-controllers-6b7b9c649d-bn8kb 0/1 Pending 0 21s controlplane: calico-node-c8cd7 0/1 Init:1/2 0 21s controlplane: calico-typha-57cbdc4588-bn6c9 1/1 Running 0 22s *********** vagrant up first run done ~~~~ ENTERING WINDOWS BRINGUP LOOP *** cni: calico calico: 3.25.0; containerd: 1.6.15 Bringing machine 'winw1' up with 'virtualbox' provider... ==> winw1: Importing base box 'sig-windows-dev-tools/windows-2019'... ==> winw1: Matching MAC address for NAT networking... ==> winw1: Checking if box 'sig-windows-dev-tools/windows-2019' version '1.0' is up to date... ==> winw1: Setting the name of the VM: sig-windows-dev-tools_winw1_1681641031922_52939 ==> winw1: Fixed port collision for 3389 => 3389. Now on port 2200. ==> winw1: Fixed port collision for 22 => 2222. Now on port 2201. ==> winw1: Clearing any previously set network interfaces... ==> winw1: Preparing network interfaces based on configuration... winw1: Adapter 1: nat winw1: Adapter 2: hostonly ==> winw1: Forwarding ports... winw1: 3389 (guest) => 2200 (host) (adapter 1) winw1: 5985 (guest) => 55985 (host) (adapter 1) winw1: 5986 (guest) => 55986 (host) (adapter 1) winw1: 22 (guest) => 2201 (host) (adapter 1) ==> winw1: Running 'pre-boot' VM customizations... ==> winw1: Booting VM... ==> winw1: Waiting for machine to boot. This may take a few minutes... winw1: WinRM address: 127.0.0.1:55985 winw1: WinRM username: vagrant winw1: WinRM execution_time_limit: PT2H winw1: WinRM transport: negotiate ==> winw1: Machine booted and ready! [winw1] No Virtualbox Guest Additions installation found. Copy iso file C:\Program Files\Oracle\VirtualBox\VBoxGuestAdditions.iso into the box $env:TEMP/VBoxGuestAdditions.iso Attached : True BlockSize : 0 DevicePath : \\.\CDROM0 FileSize : 53049344 ImagePath : C:\Users\vagrant\AppData\Local\Temp/VBoxGuestAdditions.iso LogicalSectorSize : 2048 Number : 0 Size : 53049344 StorageType : 1 PSComputerName : Mounting Virtualbox Guest Additions ISO to: D Installing Virtualbox Guest Additions 7.0.6 - guest version is unknown VBoxCertUtil.exe: info: Successfully added 'vbox-sha1.cer' to the trusted publisher store VBoxCertUtil.exe: info: Successfully added 'vbox-sha1-root.cer' to the trusted publisher store VBoxCertUtil.exe: info: Successfully added 'vbox-sha1-timestamp-root.cer' to the trusted publisher store VBoxCertUtil.exe: info: Successfully added 'vbox-sha256.cer' to the trusted publisher store VBoxCertUtil.exe: info: Successfully added 'vbox-sha256-root.cer' to the trusted publisher store VBoxCertUtil.exe: info: Successfully added 'vbox-sha256-timestamp-root.cer' to the trusted publisher store VBoxCertUtil.exe: info: Successfully added 'vbox-sha1.cer' to the root store VBoxCertUtil.exe: info: Certificate 'vbox-sha1-root.cer' is already present in the Root store and was not re-added or updated. VBoxCertUtil.exe: info: Successfully added 'vbox-sha1-timestamp-root.cer' to the root store VBoxCertUtil.exe: info: Successfully added 'vbox-sha256.cer' to the root store VBoxCertUtil.exe: info: Certificate 'vbox-sha256-root.cer' is already present in the Root store and was not re-added or updated. VBoxCertUtil.exe: info: Successfully added 'vbox-sha256-timestamp-root.cer' to the root store Unmounting Virtualbox Guest Additions ISO from: D Attached : False BlockSize : 0 DevicePath : FileSize : 53049344 ImagePath : C:\Users\vagrant\AppData\Local\Temp/VBoxGuestAdditions.iso LogicalSectorSize : 2048 Number : Size : 53049344 StorageType : 1 PSComputerName : ==> winw1: Checking for guest additions in VM... ==> winw1: Setting hostname... ==> winw1: Waiting for machine to reboot... ==> winw1: Configuring and enabling network interfaces... ==> winw1: Mounting shared folders... winw1: C:/forked => F:/Kubernetes/sig-windows-dev-tools/forked winw1: C:/sync/shared => F:/Kubernetes/sig-windows-dev-tools/sync/shared winw1: C:/sync/windows => F:/Kubernetes/sig-windows-dev-tools/sync/windows ==> winw1: Running provisioner: shell... winw1: Running: sync/windows/0-containerd.ps1 as C:\tmp\vagrant-shell.ps1 winw1: Stopping ContainerD & Kubelet winw1: Downloading Calico using ContainerD - [calico: 3.25] [containerd: 1.6.15] winw1: Installing 7Zip winw1: VERBOSE: Using the provider 'PowerShellGet' for searching packages. winw1: VERBOSE: Using the provider 'NuGet' for searching packages. winw1: VERBOSE: Total package yield:'0' for the specified package '7Zip4PowerShell'. winw1: VERBOSE: The -Repository parameter was not specified. PowerShellGet will use all of the registered repositories. winw1: VERBOSE: Getting the provider object for the PackageManagement Provider 'NuGet'. winw1: VERBOSE: The specified Location is 'https://www.powershellgallery.com/api/v2' and PackageManagementProvider is 'NuGet'. winw1: VERBOSE: Searching repository 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='7Zip4PowerShell'' for winw1: ''. winw1: VERBOSE: Total package yield:'1' for the specified package '7Zip4PowerShell'. winw1: VERBOSE: Performing the operation "Install Package" on target "Package '7Zip4Powershell' version '2.3.0' from winw1: 'PSGallery'.". winw1: VERBOSE: The installation scope is specified to be 'CurrentUser'. winw1: VERBOSE: The specified module will be installed in 'C:\Users\vagrant\Documents\WindowsPowerShell\Modules'. winw1: VERBOSE: The specified Location is 'NuGet' and PackageManagementProvider is 'NuGet'. winw1: VERBOSE: Downloading module '7Zip4Powershell' with version '2.3.0' from the repository winw1: 'https://www.powershellgallery.com/api/v2'. winw1: VERBOSE: Searching repository 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='7Zip4Powershell'' for winw1: ''. winw1: VERBOSE: InstallPackage' - name='7Zip4Powershell', winw1: version='2.3.0',destination='C:\Users\vagrant\AppData\Local\Temp\729133812' winw1: VERBOSE: DownloadPackage' - name='7Zip4Powershell', winw1: version='2.3.0',destination='C:\Users\vagrant\AppData\Local\Temp\729133812\7Zip4Powershell\7Zip4Powershell.nupkg', winw1: uri='https://www.powershellgallery.com/api/v2/package/7Zip4Powershell/2.3.0' winw1: VERBOSE: Downloading 'https://www.powershellgallery.com/api/v2/package/7Zip4Powershell/2.3.0'. winw1: VERBOSE: Completed downloading 'https://www.powershellgallery.com/api/v2/package/7Zip4Powershell/2.3.0'. winw1: VERBOSE: Completed downloading '7Zip4Powershell'. winw1: VERBOSE: Hash for package '7Zip4Powershell' does not match hash provided from the server. winw1: VERBOSE: InstallPackageLocal' - name='7Zip4Powershell', winw1: version='2.3.0',destination='C:\Users\vagrant\AppData\Local\Temp\729133812' winw1: VERBOSE: Catalog file '7Zip4Powershell.cat' is not found in the contents of the module '7Zip4Powershell' being winw1: installed. winw1: VERBOSE: Module '7Zip4Powershell' was installed successfully to path winw1: 'C:\Users\vagrant\Documents\WindowsPowerShell\Modules\7Zip4Powershell\2.3.0'. winw1: winw1: Name Version Source Summary winw1: ---- ------- ------ ------- winw1: 7Zip4Powershell 2.3.0 PSGallery Powershell module for creating and extracting 7-Zip... winw1: Getting ContainerD binaries winw1: Downloading https://github.com/containerd/containerd/releases/download/v1.6.15/containerd-1.6.15-windows-amd64.tar.gz to C:\Program Files\containerd\containerd.tar.gz winw1: x containerd-shim-runhcs-v1.exe winw1: x ctr.exe winw1: x containerd-stress.exe winw1: x containerd.exe winw1: Registering ContainerD as a service winw1: time="2023-04-16T03:47:36.862986800-07:00" level=fatal msg="The specified service already exists." winw1: Starting ContainerD service winw1: Done - please remember to add '--cri-socket "npipe:////./pipe/containerd-containerd"' to your kubeadm join command winw1: winw1: ==> winw1: Running provisioner: shell... winw1: Running: sync/windows/forked.ps1 as C:\tmp\vagrant-shell.ps1 winw1: winw1: winw1: Directory: C:\ winw1: winw1: winw1: Mode LastWriteTime Length Name winw1: ---- ------------- ------ ---- winw1: d----- 1/21/2022 3:44 AM k winw1: winw1: ==> winw1: Running provisioner: shell... winw1: Running: sync/shared/kubejoin.ps1 as C:\tmp\vagrant-shell.ps1 winw1: [preflight] Running pre-flight checks winw1: W0416 03:49:13.561149 3400 checks.go:1019] [preflight] WARNING: Couldn't create the interface used for talking to the container runtime: crictl is required by the container runtime: executable file not found in %PATH% winw1: error execution phase preflight: couldn't validate the identity of the API Server: Get "https://10.20.30.10:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s": context deadline exceeded (Client.Timeout exceeded while awaiting headers) winw1: To see the stack trace of this error execute with --v=5 or higher cni: calico calico: 3.25.0; containerd: 1.6.15 ==> winw1: Running provisioner: shell... winw1: Running: sync/windows/0-containerd.ps1 as C:\tmp\vagrant-shell.ps1 winw1: Stopping ContainerD & Kubelet winw1: Downloading Calico using ContainerD - [calico: 3.25] [containerd: 1.6.15] winw1: Installing 7Zip winw1: Getting ContainerD binaries winw1: Downloading https://github.com/containerd/containerd/releases/download/v1.6.15/containerd-1.6.15-windows-amd64.tar.gz to C:\Program Files\containerd\containerd.tar.gz winw1: x containerd-shim-runhcs-v1.exe winw1: x ctr.exe winw1: x containerd-stress.exe winw1: x containerd.exe winw1: Registering ContainerD as a service winw1: Starting ContainerD service winw1: time="2023-04-16T04:08:42.113565600-07:00" level=fatal msg="The specified service already exists." winw1: Done - please remember to add '--cri-socket "npipe:////./pipe/containerd-containerd"' to your kubeadm join command ==> winw1: Running provisioner: shell... winw1: Running: sync/windows/forked.ps1 as C:\tmp\vagrant-shell.ps1 winw1: winw1: winw1: Directory: C:\ winw1: winw1: winw1: Mode LastWriteTime Length Name winw1: ---- ------------- ------ ---- winw1: d----- 1/21/2022 3:44 AM k winw1: winw1: ==> winw1: Running provisioner: shell... winw1: Running: sync/shared/kubejoin.ps1 as C:\tmp\vagrant-shell.ps1 winw1: [preflight] Running pre-flight checks winw1: W0416 04:09:16.897024 2096 checks.go:1019] [preflight] WARNING: Couldn't create the interface used for talking to the container runtime: crictl is required by the container runtime: executable file not found in %PATH% winw1: [preflight] Reading configuration from the cluster... winw1: [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml' winw1: W0416 04:09:22.525258 2096 initconfiguration.go:119] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "npipe" to the "criSocket" with value "unix:///var/run/unknown.sock".Please update your configuration! winw1: W0416 04:09:22.535347 2096 utils.go:69] The recommended value for "authentication.x509.clientCAFile" in "KubeletConfiguration" is: \etc\kubernetes\pki\ca.crt; the provided value is: /etc/kubernetes/pki/ca.crt winw1: [kubelet-start] Writing kubelet configuration to file "\\var\\lib\\kubelet\\config.yaml" winw1: [kubelet-start] Writing kubelet environment file with flags to file "\\var\\lib\\kubelet\\kubeadm-flags.env" winw1: [kubelet-start] Starting the kubelet winw1: [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap... winw1: winw1: This node has joined the cluster: winw1: * Certificate signing request was sent to apiserver and a response was received. winw1: * The Kubelet was informed of the new secure connection details. winw1: winw1: Run 'kubectl get nodes' on the control-plane to see this node join the cluster. cni: calico cni: calico deployment.apps/nginx-deployment created service/nginx created deployment.apps/whoami-windows created service/whoami-windows created pod/netshoot created Connection to 127.0.0.1 closed. cni: calico deployment.apps/whoami-windows scaled Connection to 127.0.0.1 closed. cni: calico deployment.apps/whoami-windows scaled Connection to 127.0.0.1 closed. cni: calico Message from syslogd@ubuntu2004 at Apr 16 11:25:55 ... kernel:[ 3927.576768] watchdog: BUG: soft lockup - CPU#0 stuck for 699s! [containerd-shim:46844] Message from syslogd@ubuntu2004 at Apr 16 12:48:05 ... kernel:[ 4175.813857] watchdog: BUG: soft lockup - CPU#0 stuck for 33s! [containerd-shim:39884] Message from syslogd@ubuntu2004 at Apr 16 12:48:05 ... kernel:[ 4175.814094] watchdog: BUG: soft lockup - CPU#1 stuck for 33s! [containerd-shim:39887] pod/whoami-windows-6f7964957-gw9pt condition met pod/whoami-windows-6f7964957-k475s condition met error: timed out waiting for the condition on pods/whoami-windows-6f7964957-6rdvz Connection to 127.0.0.1 closed. make: *** [Makefile:60: 3-smoke-test] Error 1 mloskot@DEV690:/mnt/f/Kubernetes/sig-windows-dev-tools$ ```

3. Check status of nodes

Despite the make all above terminated with 3-smoke-test error the two nodes of the cluster are Ready

▶ ⎈ kubernetes-admin@kubernetes ▶ $ ▶ $ kubectl get nodes
NAME           STATUS   ROLES           AGE     VERSION
controlplane   Ready    control-plane   6h28m   v1.26.4-3+2d4a3e29be572e
winw1          Ready    <none>          5h45m   v1.26.4-3+2d4a3e29be572e

4. Check status of pods

All pods on winw1 node in namespace default have status Running

▶ ⎈ kubernetes-admin@kubernetes ▶ default ▶ $ kubectl get pods
NAME                                READY   STATUS    RESTARTS   AGE
nginx-deployment-7c6949fdf4-9sjkg   1/1     Running   0          47m
whoami-windows-6f7964957-6rdvz      1/1     Running   0          5h45m
whoami-windows-6f7964957-gw9pt      1/1     Running   0          5h45m
whoami-windows-6f7964957-k475s      1/1     Running   0          5h45m

5. Kubernetes Dashboard

I have managed to deply and access the dashboard:

  1. Follow https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/
  2. Follow https://github.com/kubernetes/dashboard/blob/master/docs/user/access-control/creating-sample-user.md
  3. Access http://127.0.0.1:8001 and authenticate with the token created in step 2.

6. Performance

I observed significant performance issues of the overall cluster. The kubectl often takes longer time to come back with results. Running kubectl get -A events --watch seems decreasing the performance even more.

I am going to restore CPU and memory tweaks from the variables.yaml patch in Successful-Run-1, that is:

 ## Linux settings
 k8s_linux_kubelet_nodeip: "10.20.30.10"
 windows_node_ip: "10.20.30.11"
-windows_ram: 6048
-linux_ram: 4096
-linux_cpus: 2
-windows_cpus: 4
+windows_ram: 8192
+linux_ram: 8192
+linux_cpus: 4
+windows_cpus: 8

Then I will perform another run to create the whole cluster.

mloskot commented 1 year ago

@aravindhp Thanks! Is /lgtm supposed to remove the do-not-merge/hold label?

aravindhp commented 1 year ago

/unhold

adeniyistephen commented 1 year ago

FWI: I was following the doc to recreate the steps, using the same host environment, but VMBox is running version 6.0. I got an error:

==> controlplane: Running 'pre-boot' VM customizations...
==> controlplane: Booting VM...
There was an error while executing `VBoxManage`, a CLI used by Vagrant
for controlling VirtualBox. The command and stderr is shown below.

Command: ["startvm", "3e926785-4d71-44a8-b942-f2357bf1bf87", "--type", "headless"]

Stderr: VBoxManage.exe: error: Failed to get device handle and/or partition ID for 00000000016c5930 (hPartitionDevice=0000000000000ad9, Last=0xc0000002/1) (VERR_NEM_VM_CREATE_FAILED)
VBoxManage.exe: error: Details: code E_FAIL (0x80004005), component ConsoleWrap, interface IConsole
make: *** [Makefile:47: 2-vagrant-up] Error 1

Apparently, I can't run VMs on VirtualBox version 6.0 while Hyper-V is enabled, so I had to upgrade to version 7.0 which supports running VMs while Hyper-V is enabled or disabled.

mloskot commented 1 year ago

@adeniyistephen Yes, I think VB 6.0 seems to be the issue. I've recalled in more details what setups I tried:

And in none of those I experienced the error as you did.

adeniyistephen commented 1 year ago

Correct @mloskot I read the development started from 6.1

k8s-ci-robot commented 1 year ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: aravindhp, mloskot

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/kubernetes-sigs/sig-windows-dev-tools/blob/master/OWNERS)~~ [aravindhp] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment