Closed jwardle closed 4 years ago
Have just discovered the below error in the Windows Event Manager. Appears the network failed to be configured. Unsure why this would be. The Windows host is a VM itself with nested virtualisation enabled and VMWare virtual ethernet adapter.
Log Name: Application
Source: docker
Date: 14/06/2019 17:03:13
Event ID: 11
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: xxxxxxxxxx
Description:
CreateProcess() failed [namespace=moby module=libcontainerd container=b34c3d30c910814bd32e2912b3269fd6d2af15785ba13da7c79ba28c6ddad645 error=container b34c3d30c910814bd32e2912b3269fd6d2af15785ba13da7c79ba28c6ddad645 encountered an error during CreateProcess: failure in a Windows system call: Unspecified error (0x80004005)
[Event Detail: failed to configure network adapter bf6ceb91-bbcd-45fd-8358-d86397615ffb: netlink.RouteAdd(netlink.Route{LinkIndex:5, ILinkIndex:0, Scope:0x0, Dst:(*net.IPNet)(nil), Src:net.IP(nil), Gw:net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0xc0, 0x0, 0x0, 0x1}, MultiPath:[]*netlink.NexthopInfo(nil), Protocol:0, Priority:1, Table:0, Type:0, Tos:0, Flags:0, MPLSDst:(*int)(nil), NewDst:netlink.Destination(nil), Encap:netlink.Encap(nil)}) failed: file exists: exit status 255 Stack Trace:
github.com/Microsoft/opengcs/service/gcs/oslayer/realos.(*realCmd).CombinedOutput
/go/src/github.com/Microsoft/opengcs/service/gcs/oslayer/realos/realos.go:127
github.com/Microsoft/opengcs/service/gcs/core/gcs.(*gcsCore).configureAdapterInNamespace
/go/src/github.com/Microsoft/opengcs/service/gcs/core/gcs/networking.go:34
github.com/Microsoft/opengcs/service/gcs/core/gcs.(*gcsCore).ExecProcess
/go/src/github.com/Microsoft/opengcs/service/gcs/core/gcs/gcs.go:369
github.com/Microsoft/opengcs/service/gcs/bridge.(*Bridge).execProcess
/go/src/github.com/Microsoft/opengcs/service/gcs/bridge/bridge.go:582
github.com/Microsoft/opengcs/service/gcs/bridge.(*Bridge).(github.com/Microsoft/opengcs/service/gcs/bridge.execProcess)-fm
/go/src/github.com/Microsoft/opengcs/service/gcs/bridge/bridge.go:236
github.com/Microsoft/opengcs/service/gcs/bridge.HandlerFunc.ServeMsg
/go/src/github.com/Microsoft/opengcs/service/gcs/bridge/bridge.go:68
github.com/Microsoft/opengcs/service/gcs/bridge.(*Mux).ServeMsg
/go/src/github.com/Microsoft/opengcs/service/gcs/bridge/bridge.go:139
github.com/Microsoft/opengcs/service/gcs/bridge.(*Bridge).ListenAfndServe.func2.1
/go/src/github.com/Microsoft/opengcs/service/gcs/bridge/bridge.go:315
runtime.goexit
/usr/lib/go/src/runtime/asm_amd64.s:2361 Provider: 00000000-0000-0000-0000-000000000000] extra info: {"CommandArgs":["dotnet","Endpoint.dll","EndpointsPath=/app/apiconfig"],"WorkingDirectory":"/app/Endpoint","Environment":{"ASPNETCORE_ENVIRONMENT":"Development","ASPNETCORE_URLS":"http://+:80","ASPNETCORE_VERSION":"2.2.5","DOTNET_RUNNING_IN_CONTAINER":"true","DOTNET_SYSTEM_GLOBALIZATION_INVARIANT":"true","HOSTNAME":"b34c3d30c910","PATH":"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"},"CreateStdInPipe":true,"CreateStdOutPipe":true,"CreateStdErrPipe":true,"ConsoleSize":[0,0],"OCISpecification":{"ociVersion":"1.0.1-dev","process":{"user":{"uid":0,"gid":0},"args":["dotnet","Endpoint.dll","EndpointsPath=/app/apiconfig"],"env":["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin","HOSTNAME=b34c3d30c910","ASPNETCORE_ENVIRONMENT=Development","ASPNETCORE_URLS=http://+:80","DOTNET_RUNNING_IN_CONTAINER=true","DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=true","ASPNETCORE_VERSION=2.2.5"],"cwd":"/app/Endpoint","capabilities":{"bounding":["CAP_CHOWN","CAP_DAC_OVERRIDE","CAP_FSETID","CAP_FOWNER","CAP_MKNOD","CAP_NET_RAW","CAP_SETGID","CAP_SETUID","CAP_SETFCAP","CAP_SETPCAP","CAP_NET_BIND_SERVICE","CAP_SYS_CHROOT","CAP_KILL","CAP_AUDIT_WRITE"],"effective":["CAP_CHOWN","CAP_DAC_OVERRIDE","CAP_FSETID","CAP_FOWNER","CAP_MKNOD","CAP_NET_RAW","CAP_SETGID","CAP_SETUID","CAP_SETFCAP","CAP_SETPCAP","CAP_NET_BIND_SERVICE","CAP_SYS_CHROOT","CAP_KILL","CAP_AUDIT_WRITE"],"inheritable":["CAP_CHOWN","CAP_DAC_OVERRIDE","CAP_FSETID","CAP_FOWNER","CAP_MKNOD","CAP_NET_RAW","CAP_SETGID","CAP_SETUID","CAP_SETFCAP","CAP_SETPCAP","CAP_NET_BIND_SERVICE","CAP_SYS_CHROOT","CAP_KILL","CAP_AUDIT_WRITE"],"permitted":["CAP_CHOWN","CAP_DAC_OVERRIDE","CAP_FSETID","CAP_FOWNER","CAP_MKNOD","CAP_NET_RAW","CAP_SETGID","CAP_SETUID","CAP_SETFCAP","CAP_SETPCAP","CAP_NET_BIND_SERVICE","CAP_SYS_CHROOT","CAP_KILL","CAP_AUDIT_WRITE"]}},"root":{"path":"rootfs"},"hostname":"b34c3d30c910","mounts":[{"destination":"/proc","type":"proc","source":"proc","options":["nosuid","noexec","nodev"]},{"destination":"/dev","type":"tmpfs","source":"tmpfs","options":["nosuid","strictatime","mode=755","size=65536k"]},{"destination":"/dev/pts","type":"devpts","source":"devpts","options":["nosuid","noexec","newinstance","ptmxmode=0666","mode=0620","gid=5"]},{"destination":"/sys","type":"sysfs","source":"sysfs","options":["nosuid","noexec","nodev","ro"]},{"destination":"/sys/fs/cgroup","type":"cgroup","source":"cgroup","options":["ro","nosuid","noexec","nodev"]},{"destination":"/dev/mqueue","type":"mqueue","source":"mqueue","options":["nosuid","noexec","nodev"]},{"destination":"/dev/shm","type":"tmpfs","source":"shm","options":["nosuid","noexec","nodev","mode=1777"]}],"linux":{"resources":{},"namespaces":[{"type":"mount"},{"type":"network"},{"type":"uts"},{"type":"pid"},{"type":"ipc"}],"maskedPaths":["/proc/asound","/proc/acpi","/proc/kcore","/proc/keys","/proc/latency_stats","/proc/timer_list","/proc/timer_stats","/proc/sched_debug","/proc/scsi","/sys/firmware"],"readonlyPaths":["/proc/bus","/proc/fs","/proc/irq","/proc/sys","/proc/sysrq-trigger"]},"windows":{"layerFolders":["C:\\ProgramData\\docker\\lcow\\4086c695b937fa060dc597c6156fd2ed907db97b3002ea391389918d34c1e9d5","C:\\ProgramData\\docker\\lcow\\7da0ac6e62a3531a15fef1f65d0cb0c1bb5f0a81d6a6a790fbb3056df14511df","C:\\ProgramData\\docker\\lcow\\92fb358790a4e4599cfe1b9bee19546836a77bc0f718794b4844ce844915d769","C:\\ProgramData\\docker\\lcow\\6da88dc63548a9bab1f045f11b7abb53747e7b199436304333017f1162d36061","C:\\ProgramData\\docker\\lcow\\36fa9d73ab7b809554d219186535977e981db91822c0fee1be479a6e349f489f","C:\\ProgramData\\docker\\lcow\\62357143c4a43d665c850a8c2fbb658bc8cc54687d54d47d7074a487e6262428","C:\\ProgramData\\docker\\lcow\\c2739f4153d622965fee14ee4826a7841a176e895fcb725be24f2b8d347030b9","C:\\ProgramData\\docker\\lcow\\febb77e5bd25dfc96da0623e3b10d5b482fa231081a0dcea8c8817ecbb6c5cbe","C:\\ProgramData\\docker\\lcow\\ec3bc1f0dcdc54caa827563dd764a5952a4e098d598e23a904b8e0068dc35312","C:\\ProgramData\\docker\\lcow\\a47aeb0cb79f49282b681f3a9592387d47d2c773c6a0163b8f2fbd926c663171","C:\\ProgramData\\docker\\lcow\\1e2efcca51719e0b7ab41d94719aa5dbb7ec37644e38baaaf79821499c74d954","C:\\ProgramData\\docker\\lcow\\d47201e626eba4c8dd102b2e891813d09f207549e32e29e2029d0ed3054c5181","C:\\ProgramData\\docker\\lcow\\419f40c2191d416c9e0fff58b6d66f86e5fd8ea22d00516bc9aed1c6212ae634","C:\\ProgramData\\docker\\lcow\\b4e804157522a99fcaab640efccca6dda3a1acea1c46a78e78259bec48d3eec6","C:\\ProgramData\\docker\\lcow\\402d3e06dc76ac5a80059608a2c900e936a3e4fefd6d3d34efb2fd9307e52876","C:\\ProgramData\\docker\\lcow\\3e6ad1e01f92a82038209142b0fb5078d6602c887f1fbc90889c51a867f91cf7","C:\\ProgramData\\docker\\lcow\\219ddfc8466f1561134832cbf734a259cee58d7edffa3d157df7282be2be8cc0","C:\\ProgramData\\docker\\lcow\\b34c3d30c910814bd32e2912b3269fd6d2af15785ba13da7c79ba28c6ddad645"],"hyperv":{},"network":{"endpointList":["AB5975E9-A692-429B-94A7-A1F27E5E7CF8","BF6CEB91-BBCD-45FD-8358-D86397615FFB"],"allowUnqualifiedDNSQuery":true}}}}]
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="docker" />
<EventID Qualifiers="0">11</EventID>
<Level>2</Level>
<Task>0</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2019-06-14T16:03:13.984131900Z" />
<EventRecordID>139479</EventRecordID>
<Channel>Application</Channel>
<Computer>xxxxxxxxx</Computer>
<Security />
</System>
<EventData>
<Data>CreateProcess() failed</Data>
<Data>namespace=moby module=libcontainerd container=b34c3d30c910814bd32e2912b3269fd6d2af15785ba13da7c79ba28c6ddad645 error=container b34c3d30c910814bd32e2912b3269fd6d2af15785ba13da7c79ba28c6ddad645 encountered an error during CreateProcess: failure in a Windows system call: Unspecified error (0x80004005)
[Event Detail: failed to configure network adapter bf6ceb91-bbcd-45fd-8358-d86397615ffb: netlink.RouteAdd(netlink.Route{LinkIndex:5, ILinkIndex:0, Scope:0x0, Dst:(*net.IPNet)(nil), Src:net.IP(nil), Gw:net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0xc0, 0x0, 0x0, 0x1}, MultiPath:[]*netlink.NexthopInfo(nil), Protocol:0, Priority:1, Table:0, Type:0, Tos:0, Flags:0, MPLSDst:(*int)(nil), NewDst:netlink.Destination(nil), Encap:netlink.Encap(nil)}) failed: file exists: exit status 255 Stack Trace:
github.com/Microsoft/opengcs/service/gcs/oslayer/realos.(*realCmd).CombinedOutput
/go/src/github.com/Microsoft/opengcs/service/gcs/oslayer/realos/realos.go:127
github.com/Microsoft/opengcs/service/gcs/core/gcs.(*gcsCore).configureAdapterInNamespace
/go/src/github.com/Microsoft/opengcs/service/gcs/core/gcs/networking.go:34
github.com/Microsoft/opengcs/service/gcs/core/gcs.(*gcsCore).ExecProcess
/go/src/github.com/Microsoft/opengcs/service/gcs/core/gcs/gcs.go:369
github.com/Microsoft/opengcs/service/gcs/bridge.(*Bridge).execProcess
/go/src/github.com/Microsoft/opengcs/service/gcs/bridge/bridge.go:582
github.com/Microsoft/opengcs/service/gcs/bridge.(*Bridge).(github.com/Microsoft/opengcs/service/gcs/bridge.execProcess)-fm
/go/src/github.com/Microsoft/opengcs/service/gcs/bridge/bridge.go:236
github.com/Microsoft/opengcs/service/gcs/bridge.HandlerFunc.ServeMsg
/go/src/github.com/Microsoft/opengcs/service/gcs/bridge/bridge.go:68
github.com/Microsoft/opengcs/service/gcs/bridge.(*Mux).ServeMsg
/go/src/github.com/Microsoft/opengcs/service/gcs/bridge/bridge.go:139
github.com/Microsoft/opengcs/service/gcs/bridge.(*Bridge).ListenAndServe.func2.1
/go/src/github.com/Microsoft/opengcs/service/gcs/bridge/bridge.go:315
runtime.goexit
/usr/lib/go/src/runtime/asm_amd64.s:2361 Provider: 00000000-0000-0000-0000-000000000000] extra info: {"CommandArgs":["dotnet","Endpoint.dll","EndpointsPath=/app/apiconfig"],"WorkingDirectory":"/app/Endpoint","Environment":{"ASPNETCORE_ENVIRONMENT":"Development","ASPNETCORE_URLS":"http://+:80","ASPNETCORE_VERSION":"2.2.5","DOTNET_RUNNING_IN_CONTAINER":"true","DOTNET_SYSTEM_GLOBALIZATION_INVARIANT":"true","HOSTNAME":"b34c3d30c910","PATH":"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"},"CreateStdInPipe":true,"CreateStdOutPipe":true,"CreateStdErrPipe":true,"ConsoleSize":[0,0],"OCISpecification":{"ociVersion":"1.0.1-dev","process":{"user":{"uid":0,"gid":0},"args":["dotnet","Endpoint.dll","EndpointsPath=/app/apiconfig"],"env":["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin","HOSTNAME=b34c3d30c910","ASPNETCORE_ENVIRONMENT=Development","ASPNETCORE_URLS=http://+:80","DOTNET_RUNNING_IN_CONTAINER=true","DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=true","ASPNETCORE_VERSION=2.2.5"],"cwd":"/app/Endpoint","capabilities":{"bounding":["CAP_CHOWN","CAP_DAC_OVERRIDE","CAP_FSETID","CAP_FOWNER","CAP_MKNOD","CAP_NET_RAW","CAP_SETGID","CAP_SETUID","CAP_SETFCAP","CAP_SETPCAP","CAP_NET_BIND_SERVICE","CAP_SYS_CHROOT","CAP_KILL","CAP_AUDIT_WRITE"],"effective":["CAP_CHOWN","CAP_DAC_OVERRIDE","CAP_FSETID","CAP_FOWNER","CAP_MKNOD","CAP_NET_RAW","CAP_SETGID","CAP_SETUID","CAP_SETFCAP","CAP_SETPCAP","CAP_NET_BIND_SERVICE","CAP_SYS_CHROOT","CAP_KILL","CAP_AUDIT_WRITE"],"inheritable":["CAP_CHOWN","CAP_DAC_OVERRIDE","CAP_FSETID","CAP_FOWNER","CAP_MKNOD","CAP_NET_RAW","CAP_SETGID","CAP_SETUID","CAP_SETFCAP","CAP_SETPCAP","CAP_NET_BIND_SERVICE","CAP_SYS_CHROOT","CAP_KILL","CAP_AUDIT_WRITE"],"permitted":["CAP_CHOWN","CAP_DAC_OVERRIDE","CAP_FSETID","CAP_FOWNER","CAP_MKNOD","CAP_NET_RAW","CAP_SETGID","CAP_SETUID","CAP_SETFCAP","CAP_SETPCAP","CAP_NET_BIND_SERVICE","CAP_SYS_CHROOT","CAP_KILL","CAP_AUDIT_WRITE"]}},"root":{"path":"rootfs"},"hostname":"b34c3d30c910","mounts":[{"destination":"/proc","type":"proc","source":"proc","options":["nosuid","noexec","nodev"]},{"destination":"/dev","type":"tmpfs","source":"tmpfs","options":["nosuid","strictatime","mode=755","size=65536k"]},{"destination":"/dev/pts","type":"devpts","source":"devpts","options":["nosuid","noexec","newinstance","ptmxmode=0666","mode=0620","gid=5"]},{"destination":"/sys","type":"sysfs","source":"sysfs","options":["nosuid","noexec","nodev","ro"]},{"destination":"/sys/fs/cgroup","type":"cgroup","source":"cgroup","options":["ro","nosuid","noexec","nodev"]},{"destination":"/dev/mqueue","type":"mqueue","source":"mqueue","options":["nosuid","noexec","nodev"]},{"destination":"/dev/shm","type":"tmpfs","source":"shm","options":["nosuid","noexec","nodev","mode=1777"]}],"linux":{"resources":{},"namespaces":[{"type":"mount"},{"type":"network"},{"type":"uts"},{"type":"pid"},{"type":"ipc"}],"maskedPaths":["/proc/asound","/proc/acpi","/proc/kcore","/proc/keys","/proc/latency_stats","/proc/timer_list","/proc/timer_stats","/proc/sched_debug","/proc/scsi","/sys/firmware"],"readonlyPaths":["/proc/bus","/proc/fs","/proc/irq","/proc/sys","/proc/sysrq-trigger"]},"windows":{"layerFolders":["C:\\ProgramData\\docker\\lcow\\4086c695b937fa060dc597c6156fd2ed907db97b3002ea391389918d34c1e9d5","C:\\ProgramData\\docker\\lcow\\7da0ac6e62a3531a15fef1f65d0cb0c1bb5f0a81d6a6a790fbb3056df14511df","C:\\ProgramData\\docker\\lcow\\92fb358790a4e4599cfe1b9bee19546836a77bc0f718794b4844ce844915d769","C:\\ProgramData\\docker\\lcow\\6da88dc63548a9bab1f045f11b7abb53747e7b199436304333017f1162d36061","C:\\ProgramData\\docker\\lcow\\36fa9d73ab7b809554d219186535977e981db91822c0fee1be479a6e349f489f","C:\\ProgramData\\docker\\lcow\\62357143c4a43d665c850a8c2fbb658bc8cc54687d54d47d7074a487e6262428","C:\\ProgramData\\docker\\lcow\\c2739f4153d622965fee14ee4826a7841a176e895fcb725be24f2b8d347030b9","C:\\ProgramData\\docker\\lcow\\febb77e5bd25dfc96da0623e3b10d5b482fa231081a0dcea8c8817ecbb6c5cbe","C:\\ProgramData\\docker\\lcow\\ec3bc1f0dcdc54caa827563dd764a5952a4e098d598e23a904b8e0068dc35312","C:\\ProgramData\\docker\\lcow\\a47aeb0cb79f49282b681f3a9592387d47d2c773c6a0163b8f2fbd926c663171","C:\\ProgramData\\docker\\lcow\\1e2efcca51719e0b7ab41d94719aa5dbb7ec37644e38baaaf79821499c74d954","C:\\ProgramData\\docker\\lcow\\d47201e626eba4c8dd102b2e891813d09f207549e32e29e2029d0ed3054c5181","C:\\ProgramData\\docker\\lcow\\419f40c2191d416c9e0fff58b6d66f86e5fd8ea22d00516bc9aed1c6212ae634","C:\\ProgramData\\docker\\lcow\\b4e804157522a99fcaab640efccca6dda3a1acea1c46a78e78259bec48d3eec6","C:\\ProgramData\\docker\\lcow\\402d3e06dc76ac5a80059608a2c900e936a3e4fefd6d3d34efb2fd9307e52876","C:\\ProgramData\\docker\\lcow\\3e6ad1e01f92a82038209142b0fb5078d6602c887f1fbc90889c51a867f91cf7","C:\\ProgramData\\docker\\lcow\\219ddfc8466f1561134832cbf734a259cee58d7edffa3d157df7282be2be8cc0","C:\\ProgramData\\docker\\lcow\\b34c3d30c910814bd32e2912b3269fd6d2af15785ba13da7c79ba28c6ddad645"],"hyperv":{},"network":{"endpointList":["AB5975E9-A692-429B-94A7-A1F27E5E7CF8","BF6CEB91-BBCD-45FD-8358-D86397615FFB"],"allowUnqualifiedDNSQuery":true}}}}</Data>
</EventData>
</Event>
@jwardle Did you get anywhere with this? We are having the same error when running Docker in a nested VM.
This issue is for https://github.com/docker/swarmkit not the old standalone Swarm.
Hi,
I have a simple stack of 1 Linux container (also built on the same server as the deployment target for now) on Windows 2019 Server with LCOW installed, configured to run Linux containers in a 2 node swarm. The containers that are being executed on this cluster are actually being built on the same machines - so not expecting any issues regarding 'platform' however if I attempt to deploy the stack without the
--resolve-image never
flag i do see anno suitable node (unsupported platform)
message. Applying the flag however removes the error message, but I see the below behaviour.docker-compose.yml
When running
docker stack deploy -f docker-compose.yml dev
the stack appears to deploy, with services and network created, however no containers themselves being started therefore seeing 0/1 replicas, I cannot retrieve logs for the service when runningdocker service logs <service name>
, nor does the Docker events or Docker daemon logs show anything of interest related to this (no errors for instance).At the current time I have no idea how to go about resolving this... any ideas would help?
If I run the imageusing
docker-compose
ordocker run
the container starts as expected.Docker Info
docker stack ps dev
docker service ls
docker service logs p1ocer8og1nx
: Returns nothing.