Docker for Windows won't start up in Linux Container mode.

szkxv6 commented 6 years ago

Hello, I have attempted to install (and reinstall) Docker CE for Windows on Windows 10 (Version 1803 Build 17134.376). Installation goes well. However on startup I receive an error shown below.

[x] I have tried with the latest version of my channel (Stable or Edge)
[x] I have uploaded Diagnostics
Diagnostics ID: 4F06A2B8-A20F-4378-A26F-C5A1C303B12A/20181108203039

Expected behavior

Docker for Windows will start up in Linux mode (Linux Containers).

Actual behavior

When starting or switching to Linux containers (Windows containers work), an error occurs as follows: Unable to create: The running command stopped because the preference variable "ErrorActionPreference" or common parameter is set to Stop: Inconsistent parameters PolicyStore PersistentStore and Dhcp Enabled

at Docker.Core.Pipe.NamedPipeClient.Send(String action, Object[] parameters) in C:\gopath\src\github.com\docker\pinata\win\src\Docker.Core\pipe\NamedPipeClient.cs:line 36 at Docker.Actions.DoStart(SynchronizationContext syncCtx, Boolean showWelcomeWindow, Boolean executeAfterStartCleanup) in C:\gopath\src\github.com\docker\pinata\win\src\Docker.Windows\Actions.cs:line 77 at Docker.Actions.<>cDisplayClass16_0.b0() in C:\gopath\src\github.com\docker\pinata\win\src\Docker.Windows\Actions.cs:line 61 at Docker.WPF.TaskQueue.<>cDisplayClass19_0.<.ctor>b1() in C:\gopath\src\github.com\docker\pinata\win\src\Docker.WPF\TaskQueue.cs:line 59

Information

Windows Version: Windows 10 (Version 1803 Build 17134.376).
Docker for Windows Version: Version 18.06.1-ce-win73 (19507) Channel: stable b0f14f1

For me, this is always reproducible. I can uninstall Docker. Install the Edge channel version. I get the same error. I can uninstall Docker. Uninstall the Hyper-V features. Restart. Reinstall Docker. Same error. Variations of this and restarts also result in this error. Windows containers do appear to work - Hello World is fine for Windows, but switching to Linux containers fails with this error.

I have seen similar errors while searching, but attempting to remove signs of the Moby VM or Hyper-V has not worked.

Most notably, I had an install that was being updated a few months back and the installation failed (after the previous version had been uninstalled through the installer). This left nothing on the machine. Maybe something gotten corrupt.

Additional Information: It looks as though the Moby VM gets set up in Hyper-V, but then suddenly gets removed. I see no noticable errors in the event viewer etc for what might have happened.

connorjga commented 5 years ago

I'd like to second this bug - I am seeing exactly the same issue. With multiple restarts of Docker and no avail, I am also unable to submit the data to collect an ID for you. Once I can successfully submit the data to you, I will post the ID.

szkxv6 commented 5 years ago

After more investigation into this - in my case at least, this appears to be more related to https://github.com/docker/for-win/issues/2050#issue-324542404

I can see that Hyper-V Switches are playing their part - as in the issue does appear to be that Docker cannot create its Virtual Switch (DockerNAT) - though it would be useful if it would just say so. It can't seem to do it because in my case, Cisco AnyConnect is stopping these extra connections. If I stop all Cisco AnyConnect services when I want to use Docker (sometimes I need to completely close and reopen Docker - a restart of Docker won't do it), it will start up fine.

I believe that I had an ocasion where I could create some other virtual switches and get Docker to start, but I made a mess of my machine when trying to reproduce it. Uninstalling or disabling the software that restricts these connections appears to be the fix.

Chances are the Cisco software uses virtual switches to control its connections and security which doesn't half interfere with Docker.

I'll carry on digging to see if I can find a way to set switches without having to disable the Cisco software - as not everyone will have those permissions.

ebriney commented 5 years ago

Hi @szkxv6 , this is a very interesting setup you have, we will soon release an enterprise version for docker desktop and I know that many companies are using AnyConnect. If you find a setup that work with AnyConnect we can at least add it in the documentation, or better find a way to detect it, or even better find a way to add rules directly in AnyConnect at install time so it works OOB. Thanks for your feedback.

szkxv6 commented 5 years ago

Hi @ebriney , I can't say for certain, but I've adjusted a couple of things and so far am seeing good results. It could somehow be a coincidence, but I haven't had this much luck before now! I have manually set my main network connection metric in an attempt to make sure that the Cisco software attempts to connect to the correct network interface (I have found that it can often try to connect to the DockerNat which causes issues) - I don't know if that in turn means that Docker has a hard time setting up the DockerNat virtual switch or not. It doesn't make much sense for a fresh Docker install, but maybe Windows can pass another default interface to Docker and cause issues.

If I get a chance, I'll try some fresh installs and see what happens.

In the mean time, what I did was...

Navigate to Control Panel\Network and Internet\Network Connections
Select my main interface (so my ideal default connection) > Properties > Advanced
I then unchecked the 'Automatic metric' box, and put a 1 in there.
I then navigated to my wireless connection, and put a 2 in the metric box (so that I had a rough default order for my main connections [1 being the first connection looked at]).

I didn't expect that to do anything on its own, but after several reboots, both Cisco AnyConnect and Docker for Windows appear to start and are useable.

If we are lucky, that is enough to get them playing nicely together. If not, I think there may still be something in those Hyper-V switches - but maybe this metric number helps with they Hyper-V interfaces - for example the default connection used in the Hyper-V switch may well change on reboots.

I suppose if anyone else has the ability to try this, let us know what happens. I'll carry on testing where I can. Regards, Adam.

ebriney commented 5 years ago

Thanks a lot @szkxv6. Good to know, in fact DockerNAT interface is used only for smb filesharing (-v). Is it still working with your setup or is it blocked?

szkxv6 commented 5 years ago

@ebriney You are right, it is blocking that. I get the response as if a firewall is blocking the connection. As soon as I disable the Cisco software, it can share the drive successfully, so it does point back to Cisco AnyConnect once more. I suppose in one way, it is doing its job, but I'm not aware if there are any settings to allow exceptions as such - maybe there are policies that can adjust what this does... but it might defeat its objective.

Another idea, which I don't know if you would really want to do or not, is to make the DockerNat switch an external switch... if it works how I'm thinking. Then the user could connect to DockerNat (or a similar named network) and it could feed both the smb and the secured office network!? (If that does work, maybe Cisco AnyConnect users can have a choice of switches if having an External connection wasn't right for any reason).

I don't think I can test that way correctly as Docker recreates the Hyper-V switch if I change it. I was able to connect to an edited Docker switch (to make it external), but I don't think Docker liked it being changed halfway through.. and then on restart it set it back again.

If you want me to try anything, I can attempt to give it a go. I vaguely remember using docker engine and setting up a seperate switch a long time in the past, would this lead us in a similar direction, or is it too unrelated to Docker for Windows?

PS - I'm back to Docker not starting... most likely because I destroyed its switch. I've managed to get it working again without a reboot - Cisco got confused and connected back to the DockerNat interface again (though it believes it is using a different Ethernet interface - the IPs just don't match). Forcing it to reconnect again, got the correct interface and fully restarting Docker (through Docker itself doesn't work), got it running again...

...surprisingly, the -v smb part is working (though I've booted an image with no data on the drives) - so maybe if the drives are shared already, it can cope.

szkxv6 commented 5 years ago

With regards to the shares, I assume the code just sees that it should be shared already so doesn't try to do it again, then when it tries to read/write to those shares, it just fails as if the files aren't there. I believe that is what I am getting now if I actually try to give it data on those shares.

Of course, this is with the Cisco software running.

aitorpazos commented 5 years ago

Hi, we are seeing similar issues with Cisco AnyConnect. Running docker containers without host volumes are working fine. However, the shared drive stops working as long as the VPN is up.

I've read reports that mentions that enabling Allow local (LAN) access when using VPN (if configured) in Cisco AnyConnect Secure Mobility Client solves the issue but I'd be concern about the security implications of that setting (and it actually has no effect on our setup, disabled by IT policies).

The workaround we are trying to agree with IT is to add 10.0.75.0/24 to Cisco's Non-Secured Routes and test it as we see that the DockerNAT routes get affected when connected to the VPN.

Routes when not connected to the VPN:

IPv4 Route Table
--
===========================================================================
Active Routes:
Network Destination        Netmask          Gateway       Interface  Metric
0.0.0.0          0.0.0.0   192.168.11.254   192.168.11.117     35
10.0.75.0    255.255.255.0         On-link         10.0.75.1    271
10.0.75.1  255.255.255.255         On-link         10.0.75.1    271
10.0.75.255  255.255.255.255         On-link         10.0.75.1    271

Routes when connected to the VPN:

IPv4 Route Table
--
===========================================================================
Active Routes:
Network Destination        Netmask          Gateway       Interface  Metric
0.0.0.0          0.0.0.0   192.168.11.254   192.168.11.117     35
0.0.0.0          0.0.0.0    192.168.132.1  192.168.132.118      2
10.0.75.0    255.255.255.0         On-link         10.0.75.1      2
10.0.75.0    255.255.255.0    192.168.132.1  192.168.132.118      2
10.0.75.1  255.255.255.255         On-link         10.0.75.1    271
10.0.75.255  255.255.255.255         On-link         10.0.75.1    271

That would explain why setting the metric to 1 makes it work again (not working for us, keeps using 2 as metric although we set 1 in the adapter settings).

ebriney commented 5 years ago

Thanks for your feedbacks @szkxv6 and @aitorpazos. @szkxv6 for the smb share, we do a nsenter in dockerd namespace and then use mount.cifs using the ip of the NAT switch (10.0.75.1), we also check with nc that 445 port is not blocked

docker-robott commented 5 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale comment. Stale issues will be closed after an additional 30d of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. /lifecycle stale

Crelex commented 5 years ago

/remove-lifecycle stale

mikeparker commented 5 years ago

Dupe of https://github.com/docker/for-win/issues/2050 but leaving this one open as it contains useful information.

docker / for-win