Creating container with publish-all flag might map to ports in exclusion range on Windows

kiview commented 3 years ago

[x] I have tried with the latest version of Docker Desktop
[x] I have tried disabling enabled experimental features
[x] I have uploaded Diagnostics
Diagnostics ID: EB0A54A8-73F5-47D0-8A3A-DFCA6AFC20BC/20210702105657

Actual behavior

Using the --publish-all flag will start at a certain high port and subsequently increment the mapped port number with each consecutive mapped port. However, following this process might lead to mapping ports that are part of the excluded port range of Windows network config.

Running netsh interface ipv4 show excludedportrange protocol=tcp in Powershell resulted in the following excluded port ranges on my machine (with nothing configure manually):

Protocol tcp Port Exclusion Ranges

Start Port    End Port
----------    --------
      1462        1462
      5357        5357
     49177       49276
     49277       49376
     49377       49476
     49477       49576
     49703       49802
     49812       49911
     50000       50059     *
     65415       65514

After a Docker restart, using the --publish-all flag will deterministically start at port 49153. This means, after mapping 25 ports after Docker restart, the excluded port range of 49177 is reached, and subsequently mapped ports using --publish-all won't be reachable anymore (although the container starts successfully).

Expected behavior

It should be possible to access the mapped ports if ports are published using the --publish-all flag.

Information

Is it reproducible? Yes
Is the problem new? I did not verify, but I think this problem did not appear with the Hyper-V backend.
Did the problem appear with an update? Potentially, but not pinned down to a specific version.
Windows Version: Windows 10 Pro, 10.0.19043 Build 19043
Docker Desktop Version: 3.5.1 (66090)
WSL2 or Hyper-V backend? WSL2
Are you running inside a virtualized Windows e.g. on a cloud server or a VM: No
Might be related to #3171

Steps to reproduce the behavior

From within WSL2, create a number of containers with --publish-all until the mapped ports end up in the excluded range:

for i in {1..30}; do (docker run --rm -d -P httpd); done

The range might be system-dependent, but for me, creating ~30 containers after Docker restart is enough to run into the excluded range.

Afterward, accessing the containers with ports mapped in the excluded range is not possible, e.g.:

$ docker ps
 docker ps
CONTAINER ID   IMAGE     COMMAND              CREATED          STATUS          PORTS                                     NAMES
7c65a1f22e6b   httpd     "httpd-foreground"   17 seconds ago   Up 16 seconds   0.0.0.0:49183->80/tcp, :::49183->80/tcp   jovial_nash
caf1bc2f0ef5   httpd     "httpd-foreground"   17 seconds ago   Up 16 seconds   0.0.0.0:49182->80/tcp, :::49182->80/tcp   fervent_brattain
3daea38867b6   httpd     "httpd-foreground"   17 seconds ago   Up 17 seconds   0.0.0.0:49181->80/tcp, :::49181->80/tcp   blissful_sammet
74b15566baaa   httpd     "httpd-foreground"   18 seconds ago   Up 17 seconds   0.0.0.0:49180->80/tcp, :::49180->80/tcp   xenodochial_jepsen
864c788d4592   httpd     "httpd-foreground"   18 seconds ago   Up 18 seconds   0.0.0.0:49179->80/tcp, :::49179->80/tcp   loving_roentgen
[...]
7661f13c3e08   httpd     "httpd-foreground"   28 seconds ago   Up 27 seconds   0.0.0.0:49155->80/tcp, :::49155->80/tcp   kind_bose

$ curl localhost:49179
curl: (7) Failed to connect to localhost port 49179: Connection refused

$ curl localhost:49155
<html><body><h1>It works!</h1></body></html>

mjeffrey commented 3 years ago

This fix would make me soooo happy. The ryuk issue hits me daily...

docker-robott commented 3 years ago

Issues go stale after 90 days of inactivity. Mark the issue as fresh with /remove-lifecycle stale comment. Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. /lifecycle stale

kiview commented 3 years ago

/remove-lifecycle stale

jefferai commented 2 years ago

I hit this a ton too, would love a fix. Definitely related to #3171

docker-robott commented 2 years ago

Issues go stale after 90 days of inactivity. Mark the issue as fresh with /remove-lifecycle stale comment. Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. /lifecycle stale

jefferai commented 2 years ago

/lifecycle frozen

jefferai commented 2 years ago

Froze it because it is still an issue and it can make usage very difficult in normal workflows.

LarsSchlieper commented 2 years ago

I encountered the same problem / could reproduce the error. By enabling Hyper-V in the Windows-Features, I was able to solve the problem for me.

kiview commented 1 year ago

@LarsSchlieper Enabling Hyper-V on Windows 11 Pro, but still using the WSL backend in Docker?

LarsSchlieper commented 1 year ago

@kiview Enabling Hyper-V on Windows 10 Pro (not 11), but still using the WSL backend in Docker. Yes.

docker / for-win