network_mode fails with a large number of ports

Description

In version 4.34.0, host networking support for mac went GA. This feature starts failing with a large number of ports (maxing out at 18 consistently using the example in this issue). It fails a bit inconsistently. In testing with our actual project, once we pass a certain threshold, all ports stop responding from the host (but the services still respond testing from inside the container). When creating a minimal test case to file an issue, more often what I see is that a subset of the ports stop responding from the host. I was able to reproduce this same behavior on a colleague's computer.

Reproduce

Run compose file specified below. The threshold where things start to have problems seems to be around ~20
- docker compose --file compose-bug.yaml up

See that not all containers are responding. You could do this manually, but this is a the script I used:

import http.client

for i in range(1, 21):
    port = 8000 + i
    conn = http.client.HTTPConnection("localhost", port)
    try:
        conn.request("GET", "/")
        response = conn.getresponse()
        print(f"Response from localhost:{port}: {response.status}")
    except Exception as e:
        print(f"Error connecting to localhost:{port}: {e}")
    finally:
        conn.close()

Here's some sample output:

Response from localhost:8001: 200
Response from localhost:8002: 200
Response from localhost:8003: 200
Error connecting to localhost:8004: [Errno 61] Connection refused
Response from localhost:8005: 200
Response from localhost:8006: 200
Response from localhost:8007: 200
Response from localhost:8008: 200
Response from localhost:8009: 200
Response from localhost:8010: 200
Response from localhost:8011: 200
Response from localhost:8012: 200
Response from localhost:8013: 200
Error connecting to localhost:8014: [Errno 61] Connection refused
Response from localhost:8015: 200
Response from localhost:8016: 200
Response from localhost:8017: 200
Response from localhost:8018: 200
Response from localhost:8019: 200
Response from localhost:8020: 200

network_mode didn't work for 4 and 14 here, but other runs would result in different failures. They do work from the inside:

root@docker-desktop:/# curl -I localhost:8004
HTTP/1.0 200 OK
...

Compose file:

services:
  "1":
    image: python:latest
    network_mode: host
    command: ["/usr/local/bin/python3", "-m", "http.server", "8001"]
  "2":
    image: python:latest
    network_mode: host
    command: ["/usr/local/bin/python3", "-m", "http.server", "8002"]
  "3":
    image: python:latest
    network_mode: host
    command: ["/usr/local/bin/python3", "-m", "http.server", "8003"]
  "4":
    image: python:latest
    network_mode: host
    command: ["/usr/local/bin/python3", "-m", "http.server", "8004"]
  "5":
    image: python:latest
    network_mode: host
    command: ["/usr/local/bin/python3", "-m", "http.server", "8005"]
  "6":
    image: python:latest
    network_mode: host
    command: ["/usr/local/bin/python3", "-m", "http.server", "8006"]
  "7":
    image: python:latest
    network_mode: host
    command: ["/usr/local/bin/python3", "-m", "http.server", "8007"]
  "8":
    image: python:latest
    network_mode: host
    command: ["/usr/local/bin/python3", "-m", "http.server", "8008"]
  "9":
    image: python:latest
    network_mode: host
    command: ["/usr/local/bin/python3", "-m", "http.server", "8009"]
  "10":
    image: python:latest
    network_mode: host
    command: ["/usr/local/bin/python3", "-m", "http.server", "8010"]
  "11":
    image: python:latest
    network_mode: host
    command: ["/usr/local/bin/python3", "-m", "http.server", "8011"]
  "12":
    image: python:latest
    network_mode: host
    command: ["/usr/local/bin/python3", "-m", "http.server", "8012"]
  "13":
    image: python:latest
    network_mode: host
    command: ["/usr/local/bin/python3", "-m", "http.server", "8013"]
  "14":
    image: python:latest
    network_mode: host
    command: ["/usr/local/bin/python3", "-m", "http.server", "8014"]
  "15":
    image: python:latest
    network_mode: host
    command: ["/usr/local/bin/python3", "-m", "http.server", "8015"]
  "16":
    image: python:latest
    network_mode: host
    command: ["/usr/local/bin/python3", "-m", "http.server", "8016"]
  "17":
    image: python:latest
    network_mode: host
    command: ["/usr/local/bin/python3", "-m", "http.server", "8017"]
  "18":
    image: python:latest
    network_mode: host
    command: ["/usr/local/bin/python3", "-m", "http.server", "8018"]
  "19":
    image: python:latest
    network_mode: host
    command: ["/usr/local/bin/python3", "-m", "http.server", "8019"]
  "20":
    image: python:latest
    network_mode: host
    command: ["/usr/local/bin/python3", "-m", "http.server", "8020"]

Expected behavior

network_mode works for all ports when a large number are used, not just a subset.

docker version

cholick@cholick-work-mbpro ~/workspace/github-scalar-tuesday/src $ docker version
Client:
 Version:           27.2.0
 API version:       1.47
 Go version:        go1.21.13
 Git commit:        3ab4256
 Built:             Tue Aug 27 14:14:45 2024
 OS/Arch:           darwin/amd64
 Context:           desktop-linux

Server: Docker Desktop 4.34.2 (167172)
 Engine:
  Version:          27.2.0
  API version:      1.47 (minimum version 1.24)
  Go version:       go1.21.13
  Git commit:       3ab5c7d
  Built:            Tue Aug 27 14:15:15 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.7.20
  GitCommit:        8fc6bcff51318944179630522a095cc9dbf9f353
 runc:
  Version:          1.1.13
  GitCommit:        v1.1.13-0-g58aa920
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client:
 Version:    27.2.0
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.16.2-desktop.1
    Path:     /Users/cholick/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.2-desktop.2
    Path:     /Users/cholick/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.34
    Path:     /Users/cholick/.docker/cli-plugins/docker-debug
  desktop: Docker Desktop commands (Alpha) (Docker Inc.)
    Version:  v0.0.15
    Path:     /Users/cholick/.docker/cli-plugins/docker-desktop
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     /Users/cholick/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.25
    Path:     /Users/cholick/.docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.5
    Path:     /Users/cholick/.docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.3.0
    Path:     /Users/cholick/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/cholick/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.13.0
    Path:     /Users/cholick/.docker/cli-plugins/docker-scout

Server:
 Containers: 55
  Running: 20
  Paused: 0
  Stopped: 35
 Images: 27
 Server Version: 27.2.0
 Storage Driver: overlayfs
  driver-type: io.containerd.snapshotter.v1
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 8fc6bcff51318944179630522a095cc9dbf9f353
 runc version: v1.1.13-0-g58aa920
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.10.4-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 14
 Total Memory: 54.92GiB
 Name: docker-desktop
 ID: c17df932-c0a7-48a8-ab99-ffae416396f3
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix:///Users/cholick/Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

Diagnostics ID

Skipped, this repros in colleagues computers as well, not specific to my install

Additional Info

No response

docker / for-mac