search

nforgeio / neonKUBE

Public NeonKUBE Kubernetes distribution related projects
https://neonkube.io
Apache License 2.0
78 stars 13 forks source link

Investigate Docker LCOW support #597

Closed jefflill closed 4 years ago

jefflill commented 5 years ago

The Docker edge releases look like they support LCOW (Linux Containters On Windows) as an experimental feature. This was demoed way back at DockerCon 2017 and may be closer to being production quality now.

I've been holding off looking into this because we're already taking arrows being on the bleeding edge of a lot of other tech although we do see weekly and perhaps even daily problems with the MobyLinuxVM losing network connectivity as well as sporadic problems mounting Windows files into containers (due to underlying SAMBA mount weirdness). These problems have been annoying but not crippling (just need to restart Docker maybe once or twice a day).

But unfortunately, the Windows 1903 feature update completely breaks the (currently) latest Docker stable version 2.0.0.3. We're seeing containers start with network connectivity and then lose this within seconds or minutes. I reverted the 1903 install and have manually disabled Windows updates.

It's probably time to really look into this.

More information:

Hmmm. It looks like Docker LCOW support requires Windows 1903: link

jefflill commented 5 years ago

I've switched to the Docker Edge 2.0.5.0 release and enabled experimental features and I also upgraded Windows to 1903. This seems to be working better that a few weeks ago but frankly, it doesn't appear that LCOW is actually working. It looks like you need to configure Docker to run Windows containers to enable LCOW.

I tried switching Docker to Windows containers and tried building some Linux images. They started to build and I didn't see any CPU activity in the DockerDesktopVM but the container build failed due to what looks like a container network issue. After switching back to Linux containers, the container image build worked. To be fair, the 2.0.5.0 releases notes does mention that there are known networking issues with Windows containers which I guess also impacts Linux containainers running when Docker is running in Windows Container mode,

The non-LCOW implementation of Docker Edge 2.0.5.0 may be working better with Windows 1903 though. I'm going to use this configuration for a while to see if the rest of the team should move onto 1903.

jefflill commented 5 years ago

We all upgraded to: Docker Edge 2.1.0.0

thaJeztah commented 5 years ago

LCOW support is still experimental, and the current implementation is not recommended for serious work as there's still many features missing, and quite some known bugs/limitations; the current will be removed at some point (Active work is ongoing to re-implement LCOW at the runtime level through containerd).

If your use-case is to run Linux containers on a Windows machine, you might be interested in the tech preview of Docker Desktop for WSL2;

jefflill commented 5 years ago

Yeah, that currently is our use case (Linux on Windows). We'll have to check out the the preview on WSL.

AmitHole commented 5 years ago

@jefflill any idea regarding below mentioned issue.

Do you know how to run Linux container on Windows (LCOW), with --network=transparent. Because I have linux container which I want to directly run on the host using the transparent driver. So I am running the linux container in the LCOW mode with transparent driver as network. With the below mentioned command.

docker run -d --network=Mytransparent --platform=linux --name dynamicui dynamicui:dev

But ran into the below mentioned issue, after running the above command -

docker run -d --network TeamedNet --platform linux --name dynamicui dynamicui:dev 47cf6329ea2fe7487245547f3f816a3a12b7d3901e4b65358814ae0f3cd36e1a docker: Error response from daemon: container 47cf6329ea2fe7487245547f3f816a3a12b7d3901e4b65358814ae0f3cd36e1a encountered an error during CreateProcess: failure in a Windows system call: Unspecified error (0x80004005) [Event Detail: failed to link resolv.conf file for adapter de3394a3-4b6d-4cb9-9e19-04307b13390b: link /etc/resolv.conf /tmp/base/etc/resolv.conf: invalid cross-device link Stack Trace: github.com/Microsoft/opengcs/service/gcs/oslayer/realos.(realOS).Link /go/src/github.com/Microsoft/opengcs/service/gcs/oslayer/realos/realos.go:232 github.com/Microsoft/opengcs/service/gcs/core/gcs.(gcsCore).configureAdapterInNamespace /go/src/github.com/Microsoft/opengcs/service/gcs/core/gcs/networking.go:57 github.com/Microsoft/opengcs/service/gcs/core/gcs.(gcsCore).ExecProcess /go/src/github.com/Microsoft/opengcs/service/gcs/core/gcs/gcs.go:369 github.com/Microsoft/opengcs/service/gcs/bridge.(Bridge).execProcess /go/src/github.com/Microsoft/opengcs/service/gcs/bridge/bridge.go:582 github.com/Microsoft/opengcs/service/gcs/bridge.(Bridge).(github.com/Microsoft/opengcs/service/gcs/bridge.execProcess)-fm /go/src/github.com/Microsoft/opengcs/service/gcs/bridge/bridge.go:236 github.com/Microsoft/opengcs/service/gcs/bridge.HandlerFunc.ServeMsg /go/src/github.com/Microsoft/opengcs/service/gcs/bridge/bridge.go:68 github.com/Microsoft/opengcs/service/gcs/bridge.(Mux).ServeMsg /go/src/github.com/Microsoft/opengcs/service/gcs/bridge/bridge.go:139 github.com/Microsoft/opengcs/service/gcs/bridge.(*Bridge).ListenAndServe.func2.1 /go/src/github.com/Microsoft/opengcs/service/gcs/bridge/bridge.go:315 runtime.goexit /usr/lib/go/src/runtime/asm_amd64.s:2361 Provider: 00000000-0000-0000-0000-000000000000] extra info: {"CommandArgs":["/bin/sh","-c","./Cognex.DynamicUIManager.Server"],"WorkingDirectory":"/DynamicUIManager","Environment":{"ASPNETCORE_URLS":"http://+:80","ASPNETCORE_VERSION":"2.2.6","DOTNET_RUNNING_IN_CONTAINER":"true","HOSTNAME":"47cf6329ea2f","PATH":"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"},"CreateStdInPipe":true,"CreateStdOutPipe":true,"CreateStdErrPipe":true,"ConsoleSize":[0,0],"OCISpecification":{"ociVersion":"1.0.1-dev","process":{"user":{"uid":0,"gid":0},"args":["/bin/sh","-c","./Cognex.DynamicUIManager.Server"],"env":["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin","HOSTNAME=47cf6329ea2f","ASPNETCORE_URLS=http://+:80","DOTNET_RUNNING_IN_CONTAINER=true","ASPNETCORE_VERSION=2.2.6"],"cwd":"/DynamicUIManager","capabilities":{"bounding":["CAP_CHOWN","CAP_DAC_OVERRIDE","CAP_FSETID","CAP_FOWNER","CAP_MKNOD","CAP_NET_RAW","CAP_SETGID","CAP_SETUID","CAP_SETFCAP","CAP_SETPCAP","CAP_NET_BIND_SERVICE","CAP_SYS_CHROOT","CAP_KILL","CAP_AUDIT_WRITE"],"effective":["CAP_CHOWN","CAP_DAC_OVERRIDE","CAP_FSETID","CAP_FOWNER","CAP_MKNOD","CAP_NET_RAW","CAP_SETGID","CAP_SETUID","CAP_SETFCAP","CAP_SETPCAP","CAP_NET_BIND_SERVICE","CAP_SYS_CHROOT","CAP_KILL","CAP_AUDIT_WRITE"],"inheritable":["CAP_CHOWN","CAP_DAC_OVERRIDE","CAP_FSETID","CAP_FOWNER","CAP_MKNOD","CAP_NET_RAW","CAP_SETGID","CAP_SETUID","CAP_SETFCAP","CAP_SETPCAP","CAP_NET_BIND_SERVICE","CAP_SYS_CHROOT","CAP_KILL","CAP_AUDIT_WRITE"],"permitted":["CAP_CHOWN","CAP_DAC_OVERRIDE","CAP_FSETID","CAP_FOWNER","CAP_MKNOD","CAP_NET_RAW","CAP_SETGID","CAP_SETUID","CAP_SETFCAP","CAP_SETPCAP","CAP_NET_BIND_SERVICE","CAP_SYS_CHROOT","CAP_KILL","CAP_AUDIT_WRITE"]}},"root":{"path":"rootfs"},"hostname":"47cf6329ea2f","mounts":[{"destination":"/proc","type":"proc","source":"proc","options":["nosuid","noexec","nodev"]},{"destination":"/dev","type":"tmpfs","source":"tmpfs","options":["nosuid","strictatime","mode=755","size=65536k"]},{"destination":"/dev/pts","type":"devpts","source":"devpts","options":["nosuid","noexec","newinstance","ptmxmode=0666","mode=0620","gid=5"]},{"destination":"/sys","type":"sysfs","source":"sysfs","options":["nosuid","noexec","nodev","ro"]},{"destination":"/sys/fs/cgroup","type":"cgroup","source":"cgroup","options":["ro","nosuid","noexec","nodev"]},{"destination":"/dev/mqueue","type":"mqueue","source":"mqueue","options":["nosuid","noexec","nodev"]},{"destination":"/dev/shm","type":"tmpfs","source":"shm","options":["nosuid","noexec","nodev","mode=1777"]}],"linux":{"resources":{},"namespaces":[{"type":"mount"},{"type":"network"},{"type":"uts"},{"type":"pid"},{"type":"ipc"}],"maskedPaths":["/proc/asound","/proc/acpi","/proc/kcore","/proc/keys","/proc/latency_stats","/proc/timer_list","/proc/timer_stats","/proc/sched_debug","/proc/scsi","/sys/firmware"],"readonlyPaths":["/proc/bus","/proc/fs","/proc/irq","/proc/sys","/proc/sysrq-trigger"]},"windows":{"layerFolders":["C:\ProgramData\Docker\lcow\ce5656471f1972d1fb065863f34ca3d631c431f3fad97d77ff6fa6fd4bfb5f9b","C:\ProgramData\Docker\lcow\73c2b07e7168f0701ed45af4933f92ce9888a0b556014fd267d2d8f473618ed2","C:\ProgramData\Docker\lcow\7cb5da704e42d7ad33560322d8a69a2f59811501f288e5c7c238be792678a205","C:\ProgramData\Docker\lcow\284964a9ea03c7b4c044df2c54737000596d144a32c200a5dcf1d6a520abfbc5","C:\ProgramData\Docker\lcow\2657b3efd2cebb278566ff8a523aa8c9d7b83a841de626f461b5513a55b66092","C:\ProgramData\Docker\lcow\b1b031fdab0ab0279f303a8540a7eb04cf27bae2a08b1a8d82640b2c3601e79c","C:\ProgramData\Docker\lcow\5b8011f4e4693c51cd42000cc6afc6e127173f63a1746ee6c942487fe634d605","C:\ProgramData\Docker\lcow\2500e1d0c12cc21339dd2ae544337a08c38581afe160e0f7dd1bd4693f7378c3","C:\ProgramData\Docker\lcow\47cf6329ea2fe7487245547f3f816a3a12b7d3901e4b65358814ae0f3cd36e1a"],"hyperv":{},"network":{"endpointList":["de3394a3-4b6d-4cb9-9e19-04307b13390b"]}}}}.

jefflill commented 4 years ago

Recent Docker releases have done this integration with updated standard Windows builds, and is much more reliable now.

I haven’t seen any issues for months now. I’m going to close this.