Open hypeitnow opened 2 years ago
Hello, thanks for the bug report! Can you help me understand your setup better? I'm confused about your use of cgroups v1 versus v2. Which one is default? Is there a reason that you modified which version you are using? It seems to me like the above description is for a problem that occurs after trying to work around the original problem, rather than describing the original problem. Is this true? Any help understanding this is much appreciated.
Thank you for your reply in the first place, it is not aimed at resolving any problems, I just wanted to switch to cgroups v2 from v1 since it's beneficial for the reasons described e.g here https://thenewstack.io/linux-cgroups-v2-brings-rootless-containers-superior-memory-management/ Just to make sure I switched wsl instance back to cgroups v1 but the issue I mentioned when trying to build containers stille persists WARN[0000] lstat /root/Dockerfile: no such file or directory, using Containerfile as fallback FATA[0000] lstat /root/Containerfile: no such file or directory
That's good to know. Is there anything else that is atypical about your environment? It's difficult to fix the problem when we can't reproduce it :)
I have the same issue. I've been trying to get K8s running with Docker, but only had partial success, so I decided to start over with a clean image and try nerdctl to see if that worked any better.
My setup is WSL2, Ubuntu image 22.04. Same OS, Rancher Desktop and K8s versions. I wanted to keep it clean, so I didn't mess with the configuration, just updated.
Given the issues I had previously, install seemed to work best when I had shutdown WSL, so I did that before running the installer. Docker Desktop for Windows was not running during install, or after.
ifconfig showed the following and I was able to ping all of the IPv4 addresses, which is a good sign:
johnl@DESKTOP-FDQ7CV5:~$ ifconfig
cni0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet 10.42.0.1 netmask 255.255.255.0 broadcast 10.42.0.255
inet6 fe80::24a4:48ff:fe03:8d74 prefixlen 64 scopeid 0x20<link>
ether 26:a4:48:03:8d:74 txqueuelen 1000 (Ethernet)
RX packets 7799 bytes 1590224 (1.5 MB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 8997 bytes 2390098 (2.3 MB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.21.211.86 netmask 255.255.240.0 broadcast 172.21.223.255
inet6 fe80::215:5dff:fe57:df78 prefixlen 64 scopeid 0x20<link>
ether 00:15:5d:57:df:78 txqueuelen 1000 (Ethernet)
RX packets 36032 bytes 48580939 (48.5 MB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 5730 bytes 4888506 (4.8 MB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
flannel.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet 10.42.0.0 netmask 255.255.255.255 broadcast 0.0.0.0
inet6 fe80::8c21:34ff:fe02:bbf3 prefixlen 64 scopeid 0x20<link>
ether 8e:21:34:02:bb:f3 txqueuelen 0 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 5 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1000 (Local Loopback)
RX packets 24491 bytes 9588503 (9.5 MB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 24491 bytes 9588503 (9.5 MB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
veth19feb9db: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet6 fe80::f0c7:5ff:fec4:8363 prefixlen 64 scopeid 0x20<link>
ether f2:c7:05:c4:83:63 txqueuelen 0 (Ethernet)
RX packets 1331 bytes 126849 (126.8 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1432 bytes 151731 (151.7 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
veth861b7861: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet6 fe80::c8f:bff:fec0:6e9 prefixlen 64 scopeid 0x20<link>
ether 0e:8f:0b:c0:06:e9 txqueuelen 0 (Ethernet)
RX packets 70 bytes 6594 (6.5 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 141 bytes 17002 (17.0 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vethb280045d: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet6 fe80::242b:cfff:fe97:e683 prefixlen 64 scopeid 0x20<link>
ether 26:2b:cf:97:e6:83 txqueuelen 0 (Ethernet)
RX packets 4297 bytes 1104056 (1.1 MB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 5137 bytes 821957 (821.9 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vethb8af9501: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet6 fe80::1489:1aff:fe2d:9364 prefixlen 64 scopeid 0x20<link>
ether 16:89:1a:2d:93:64 txqueuelen 0 (Ethernet)
RX packets 1637 bytes 112622 (112.6 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1667 bytes 114870 (114.8 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vethf6daaa16: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet6 fe80::e4c1:4cff:fe22:b51b prefixlen 64 scopeid 0x20<link>
ether e6:c1:4c:22:b5:1b txqueuelen 0 (Ethernet)
RX packets 1223 bytes 106457 (106.4 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1756 bytes 215304 (215.3 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
The installation instructions indicate that Helm, kubectl, nerdctl, Moby and Docker Compose will be available after install. When I ran kubectl version, it wasn't found and downloaded the latest version (I hope this doesn't cause issues). Here's the output when I checked on the other apps:
johnl@DESKTOP-FDQ7CV5:~/hello-world$ nerdctl build --tag helloworld:v1.0 .
WARN[0000] lstat /sys/fs/cgroup/rancher-desktop/run/nerdctl-tmp.986227269/input.1118623100/Dockerfil
e: no such file or directory, using Containerfile as fallback
FATA[0000] lstat /sys/fs/cgroup/rancher-desktop/run/nerdctl-tmp.986227269/input.1118623100/Container
file: no such file or directory
johnl@DESKTOP-FDQ7CV5:~/hello-world$ helm version
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /home/johnl/.kube/config
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /home/johnl/.kube/config
version.BuildInfo{Version:"v3.9.1", GitCommit:"a7c043acb5ff905c261cfdc923a35776ba5e66e4", GitTreeState:"clean", GoVersion:"go1.17.5"}
johnl@DESKTOP-FDQ7CV5:~/hello-world$ moby version
Command 'moby' not found, did you mean:
command 'goby' from deb goby-java (3.3.1+dfsg2-7)
Try: sudo apt install <deb name>
johnl@DESKTOP-FDQ7CV5:~/hello-world$ docker version
The command 'docker' could not be found in this WSL 2 distro.
We recommend to activate the WSL integration in Docker Desktop settings.
For details about using Docker Desktop with WSL 2, visit:
https://docs.docker.com/go/wsl2/
johnl@DESKTOP-FDQ7CV5:~/hello-world$ docker-compose version
The command 'docker-compose' could not be found in this WSL 2 distro.
We recommend to activate the WSL integration in Docker Desktop settings.
For details about using Docker Desktop with WSL 2, visit:
https://docs.docker.com/go/wsl2/
johnl@DESKTOP-FDQ7CV5:~$ containerd version
Command 'containerd' not found, but can be installed with:
sudo apt install containerd
Here are the results when I check for the apps in rancher-desktop:
~ # helm version
version.BuildInfo{Version:"v3.9.1", GitCommit:"a7c043acb5ff905c261cfdc923a35776ba5e66e4", GitTreeState:"clean", GoVersion:"go1.17.5"}
~ # nerdctl version
Client:
Version: v0.22.2
OS/Arch: linux/amd64
Git commit: 2899222cb0715f1e5ffe356d10c3439ee8ee3ba4
buildctl:
Version: v0.10.3
GitCommit: c8d25d9a103b70dc300a4fd55e7e576472284e31
FATA[0000] cannot access containerd socket "/run/containerd/containerd.sock": no such file or directory
~ # moby version
-sh: moby: not found
~ # docker version
Client:
Version: 20.10.16
API version: 1.41
Go version: go1.18.5
Git commit: aa7e414fdcb23a66e8fabbef0a560ef1769eace5
Built: Tue Aug 2 11:08:17 2022
OS/Arch: linux/amd64
Context: default
Experimental: true
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
~ # docker-compose version
The command 'docker-compose' could not be found in this WSL 2 distro.
We recommend to activate the WSL integration in Docker Desktop settings.
For details about using Docker Desktop with WSL 2, visit:
https://docs.docker.com/go/wsl2/
~ # kubectl version
I0830 19:26:05.329229 7689 versioner.go:58] invalid configuration: no configuration has been provided
I0830 19:26:05.333519 7689 versioner.go:64] No local kubectl binary found, fetching latest stable release version
I0830 19:26:05.627855 7689 versioner.go:84] Right kubectl missing, downloading version 1.25.0
Downloading https://storage.googleapis.com/kubernetes-release/release/v1.25.0/bin/linux/amd64/kubectl
kubectl1.25.0 100% |████████████████████████████████████████| (45/45 MB, 29.381 MB/s) [1s:0s] done.
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short. Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.0", GitCommit:"a866cbe2e5bbaa01cfd5e969aa3e033f3282a8a2", GitTreeState:"clean", BuildDate:"2022-08-23T17:44:59Z", GoVersion:"go1.19", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.7
The connection to the server localhost:8080 was refused - did you specify the right host or port?
(NOTE: I didn't get the localhost error after kubectl was installed on Ubuntu; everything else was the same)
Do we need to copy, link, or manually install these apps? Please provide details.
Thanks, John
@sree792 thanks for your input! It is expected that kubectl
will download on the first run - it is actually a symlink to kuberlr, which ensures that you have the correct version of kubectl
for the version of the k8s API server you're running.
Not sure what the log about the connection to localhost:8080 being refused is about. Perhaps you didn't wait until everything was up? This is probably a red herring, and is not related to the topic of this issue anyways.
Thanks for responding.
Am I supposed to access K3s from my Ubuntu distro, the rancher-desktop distro, or both?
The install instructions aren't clear whether WSL2 and the distro's should be active prior to the install (but implies that WSL2 will be configured as a part of the set-up). I'm wondering if my issue is related to system configuration during installation.
re: connection to localhost: kubectl doesn't have that issue in the Ubuntu distro. In summary, the network is in better shape in Ubuntu; the RD distro appears to have more of the apps installed, but some have configuration issues and others are missing.
FYI, I also posted this on Slack, but I'm happy to communicate in either forum. Here's a link to the thread and an update from there: https://rancher-users.slack.com/archives/C0200L1N1MM/p1661916169255019?thread_ts=1661889117.612039&cid=C0200L1N1MM
Doug Merrill Are you running the rancher-desktop app during your testing? The links required are setup as the app initializes. If you are running the app, are you seeing any errors on the app?
johnl800 Yes, after install the rancher GUI shows that the Helm charts are complete and all services are active.
When rancher is down then the rancher-desktop distro in WSL2 is inactive, so I would not have been able to repeat the tests there. Notice that nerdctl and Docker are both available in the rancher distro, but not in Ubuntu. Also, it doesn't appear that containerd is available in either distro, which is likely, the cause of the issue.
I'm hoping for a solution from someone who knows how to solve the issue properly, instead of cobbing a solution myself.
Am I supposed to access K3s from my Ubuntu distro, the rancher-desktop distro, or both?
On Windows, the intended use of RD is from your own (i.e. not the rancher-desktop distro) WSL distro or from the regular Windows machine. We have things like rdctl
because some workarounds require that the user goes into the rancher-desktop WSL distro, and some people are just curious and want to tweak things. You shouldn't be going into the rancher-desktop distro for your day-to-day work.
The install instructions aren't clear whether WSL2 and the distro's should be active prior to the install (but implies that WSL2 will be configured as a part of the set-up). I'm wondering if my issue is related to system configuration during installation.
I don't think that's likely, though maybe. RD should take whatever state the system is in at install time and either produce an error if the setup won't work, or make any needed changes if possible.
I'm hoping for a solution from someone who knows how to solve the issue properly, instead of cobbing a solution myself.
Unfortunately we haven't run into this before. And we have limited bandwidth, along with a need to deliver new features and fix other users' issues. If you're able to solve your issue, please post here so that others can benefit. Otherwise, we'll get to this as soon as we can :smile:
Hi Adam, I know that you're busy and I appreciate the time that you've spent to support me.
I'm new to WSL and K8s, so I'm still getting a handle on the details of the components, plumbing, etc. My intuition is that RD didn't install/link all of the components that were intended in the RD dist and/or my Ubuntu dist. I can install/link them manually, but I'm hoping that someone can provide the correct configuration (e.g. "containerd is installed in dist x:path y, with a sim-links in dist x:path z, dist a:path b"). Then I can correct the configuration without breaking the intended integration.
Can you provide that information for nerdctl and containerd?
Thanks, John
Actual Behavior
Steps to Reproduce
Result
No image is created.
Expected Behavior
Running nerdctl build from the tutorial https://docs.rancherdesktop.io/how-to-guides/hello-world-example fails, the expected bahaviour is to build Docker image.
Additional Information
The WSL distribution is currently using cgroups v1 since the
kernelCommandLine = cgroup_no_v1=all
option in .wslconfig file and mounting cgroups v2mount -t cgroup2 cgroup2 /sys/fs/cgroup -o rw,nosuid,nodev,noexec,relatime,nsdelegate
causes nerdctl to produce following errors:Rancher Desktop Version
1.5.1
Rancher Desktop K8s Version
1.2.4
Which container engine are you using?
containerd (nerdctl)
What operating system are you using?
Windows
Operating System / Build Version
Windows version 21H2 19044.1889
What CPU architecture are you using?
x64
Linux only: what package format did you use to install Rancher Desktop?
No response
Windows User Only
System Information
McAfee Agent
Version number: 5.7.4.399
McAfee Endpoint Security
Version number: 10.7 For module and content information, view the About page
in the McAfee Endpoint Security Client
Copyright (C) 2021 McAfee LLC.
All Rights Reserved.
www.mcafee.com