search

microsoft / vscode-remote-release

Visual Studio Code Remote Development: Open any folder in WSL, in a Docker container, or on a remote machine using SSH and take advantage of VS Code's full feature set.
https://aka.ms/vscode-remote
Other
3.67k stars 289 forks source link

Tunnels to incorrect port number (local port + 1) #8154

Open PaulBurridge opened 1 year ago

PaulBurridge commented 1 year ago

When any port is opened in container it is auto forwarded to a different port (source port + 1)

Steps to Reproduce:

1) Install netcat sudo apt install netcat -y 2) Execute netcat -l 6050 (any port number will do) 3) Tunnel is created to localhost:6051

image

If I stop port forward and manually forward it uses the correct port.

image

image

Original issue was using flask with Python on port 5000 (which gets tunneled to 5001). The software I am overdeveloping does not work when using a different port on the host. (images etc link to wrong port)

I have tried numerous ports all have same behavior.

This issue has only surfaced in the last 48hrs, tried rebooting PC, reinstall of WSL and recreation of container which make no difference.

chrmarti commented 1 year ago

Please check if the local port is in use, we should use the same port if that is available. Are you by any chance using host networking for the container? If not, please append the devcontainer.json (and any Dockerfile and docker-compose.yml) you are using.

PaulBurridge commented 1 year ago

Hi the port is free on the host and WSL, and to be clear this happens for any port that I have tried not just the ones listed above.

I am not using a devcontainer.json but rather a image based on the standard Python Container - https://github.com/docker-library/python/blob/f568f56f28fab0fe87b34db777e2c2861cef002b/3.9/bullseye/Dockerfile, with just a few extra apps installed via apt.

I am starting the container from a WSL instance with command Docker Containers: Start the using Dev Containers: Attach to Running Conatiner...

This image has been working for the past year with no issues.

PaulBurridge commented 1 year ago

Also I am not using host-networking. I have attached the container to a docker network created via docker compose.

PaulBurridge commented 1 year ago

Tried the following with a clean new container:

in WSL docker run -it --rm python:3.9-bullseye

Dev Containers: Attach to Running Conatainer...

In the container

apt-get update && apt-get install netcat -y netcat -l 5123

image


[
    {
        "Id": "50cff3ff92718c405731697872680432abeb1c33d4f33a321b860cd9bd553359",
        "Created": "2023-03-08T09:04:14.17904608Z",
        "Path": "python3",
        "Args": [],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 32125,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2023-03-08T09:04:15.544655978Z",
            "FinishedAt": "0001-01-01T00:00:00Z"
        },
        "Image": "sha256:cada33548249c01bdd312ce8bce6260fe77189e28d1342f31b0c086993e25156",
        "ResolvConfPath": "/var/lib/docker/containers/50cff3ff92718c405731697872680432abeb1c33d4f33a321b860cd9bd553359/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/50cff3ff92718c405731697872680432abeb1c33d4f33a321b860cd9bd553359/hostname",
        "HostsPath": "/var/lib/docker/containers/50cff3ff92718c405731697872680432abeb1c33d4f33a321b860cd9bd553359/hosts",
        "LogPath": "/var/lib/docker/containers/50cff3ff92718c405731697872680432abeb1c33d4f33a321b860cd9bd553359/50cff3ff92718c405731697872680432abeb1c33d4f33a321b860cd9bd553359-json.log",
        "Name": "/heuristic_mahavira",
        "RestartCount": 0,
        "Driver": "overlay2",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": [
            "e21447f5771559eeef739a9f85eac9b643afdbeaa15efa2427b8d744e5f2240f",
            "3939df9ac90d1595dcb4ef511b8d8879294a15ea44aaaab78a8e5b9e0c18c8ce",
            "5f5cbd98583ed9005f3b7e0e3e655439c9ac3b24bba21766986d040845c25b75",
            "8a46a76e4ac5377702a6202d175332ecac66890a093ddee7abb6279e0cfde71c",
            "cc366d214e4f56e89f0f85f1cac8c0728a942af0facaa339c34f3360994a63d0",
            "072cb8655a093409b62e7a83289431b217a6fc336a7406a6046979f8f22e343d",
            "36c1c55e552cb0f6a392a857a0a0e53b4b077f70a031d21b1713a120b622fd29"
        ],
        "HostConfig": {
            "Binds": null,
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "default",
            "PortBindings": {},
            "RestartPolicy": {
                "Name": "no",
                "MaximumRetryCount": 0
            },
            "AutoRemove": true,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "ConsoleSize": [
                27,
                201
            ],
            "CapAdd": null,
            "CapDrop": null,
            "CgroupnsMode": "host",
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "private",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": [],
            "BlkioDeviceReadBps": [],
            "BlkioDeviceWriteBps": [],
            "BlkioDeviceReadIOps": [],
            "BlkioDeviceWriteIOps": [],
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DeviceCgroupRules": null,
            "DeviceRequests": null,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": false,
            "PidsLimit": null,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": [
                "/proc/asound",
                "/proc/acpi",
                "/proc/kcore",
                "/proc/keys",
                "/proc/latency_stats",
                "/proc/timer_list",
                "/proc/timer_stats",
                "/proc/sched_debug",
                "/proc/scsi",
                "/sys/firmware"
            ],
            "ReadonlyPaths": [
                "/proc/bus",
                "/proc/fs",
                "/proc/irq",
                "/proc/sys",
                "/proc/sysrq-trigger"
            ]
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/7334b66d01a03c8a4ee48a967847adc89f34b7a86c0fd11a96147b7a4220bb28-init/diff:/var/lib/docker/overlay2/ebab21281697f626cb4ee39a72e6531e0bb42b1c52d463d8b46f4ade37e48ef4/diff:/var/lib/docker/overlay2/0116d5c733940ebdb5c3f821c72bc41043287ab3cf2eded53f69307f1dc967fd/diff:/var/lib/docker/overlay2/8b7834943975a4c32f292ceb38f9938f50904bc70980f37439b52844c0bb1d34/diff:/var/lib/docker/overlay2/573e473122183bc25c0dba994af8aac2dcc6c38b78088d645bd77fb374470141/diff:/var/lib/docker/overlay2/4d512d4eb19985e16175c63741124146ad49811a00d038feb8be1cdd2600356f/diff:/var/lib/docker/overlay2/ea7edfef8d2fb8237591944463801e31521f2c496d1820479f569dd89b18a135/diff:/var/lib/docker/overlay2/8e5713be7c725c7080540a5460021e896e934dde569de481ee6085c05ddb1fb0/diff:/var/lib/docker/overlay2/e8a4f732ddb874b08a2dc23af9a2fa73ca19e01440465d980653994f28431b52/diff:/var/lib/docker/overlay2/871b7e23c10fdf3113f1a86c1e3ed934757a17ac2b73cbb3dde14a97c31f610f/diff",
                "MergedDir": "/var/lib/docker/overlay2/7334b66d01a03c8a4ee48a967847adc89f34b7a86c0fd11a96147b7a4220bb28/merged",
                "UpperDir": "/var/lib/docker/overlay2/7334b66d01a03c8a4ee48a967847adc89f34b7a86c0fd11a96147b7a4220bb28/diff",
                "WorkDir": "/var/lib/docker/overlay2/7334b66d01a03c8a4ee48a967847adc89f34b7a86c0fd11a96147b7a4220bb28/work"
            },
            "Name": "overlay2"
        },
        "Mounts": [],
        "Config": {
            "Hostname": "50cff3ff9271",
            "Domainname": "",
            "User": "",
            "AttachStdin": true,
            "AttachStdout": true,
            "AttachStderr": true,
            "Tty": true,
            "OpenStdin": true,
            "StdinOnce": true,
            "Env": [
                "PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "LANG=C.UTF-8",
                "GPG_KEY=E3FF2839C048B25C084DEBE9B26995E310250568",
                "PYTHON_VERSION=3.9.16",
                "PYTHON_PIP_VERSION=22.0.4",
                "PYTHON_SETUPTOOLS_VERSION=58.1.0",
                "PYTHON_GET_PIP_URL=https://github.com/pypa/get-pip/raw/d5cb0afaf23b8520f1bbcfed521017b4a95f5c01/public/get-pip.py",
                "PYTHON_GET_PIP_SHA256=394be00f13fa1b9aaa47e911bdb59a09c3b2986472130f30aa0bfaf7f3980637"
            ],
            "Cmd": [
                "python3"
            ],
            "Image": "python:3.9-bullseye",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {}
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "5fe5ddb5eccad23335e5d8a1eceef4ce70f05af9f34e9af8eba5fc6cd290871d",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {},
            "SandboxKey": "/var/run/docker/netns/5fe5ddb5ecca",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "ff90c2de521669d9411642180ac269f40c516a79568d34d75ca21084d7c7fcc6",
            "Gateway": "172.17.0.1",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "172.17.0.3",
            "IPPrefixLen": 16,
            "IPv6Gateway": "",
            "MacAddress": "02:42:ac:11:00:03",
            "Networks": {
                "bridge": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "3138573298bfec620cf5dfecac1d7c1d2c16773d7869c2797559916b466368b2",
                    "EndpointID": "ff90c2de521669d9411642180ac269f40c516a79568d34d75ca21084d7c7fcc6",
                    "Gateway": "172.17.0.1",
                    "IPAddress": "172.17.0.3",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:11:00:03",
                    "DriverOpts": null
                }
            }
        }
    }
]```
PaulBurridge commented 1 year ago

Just to rule out container did same with alpine

image

PaulBurridge commented 1 year ago

So did some tests (server on port 5000) and find the following sequence of events:

It appears wslhost.exe grabs port 5000 for a while forcing vscode to use port 5001.

Evidence:

Checking ports from windows before

C:\Users\paul.burridge>netstat -ano | find "LISTENING" | find "50"
  TCP    0.0.0.0:5040           0.0.0.0:0              LISTENING       10800

The run server in container

image

Checking ports from windows 

C:\Users\paul.burridge>netstat -ano | find "LISTENING" | find "50"
  TCP    0.0.0.0:5040           0.0.0.0:0              LISTENING       10800
  TCP    127.0.0.1:5000         0.0.0.0:0              LISTENING       22972
  TCP    127.0.0.1:5001         0.0.0.0:0              LISTENING       12160

C:\Users\paul.burridge>tasklist /fi "PID eq 22972"

Image Name                     PID Session Name        Session#    Mem Usage
========================= ======== ================ =========== ============
wslhost.exe                  22972 Console                    3      7,968 K

C:\Users\paul.burridge>tasklist /fi "PID eq 12160"

Image Name                     PID Session Name        Session#    Mem Usage
========================= ======== ================ =========== ============
Code.exe                     12160 Console                    3     96,372 K

On WSL

sudo ss -tulpn | grep LISTEN
tcp   LISTEN 0      4096         0.0.0.0:3000       0.0.0.0:*    users:(("docker-proxy",pid=64216,fd=4))
tcp   LISTEN 0      511        127.0.0.1:39963      0.0.0.0:*    users:(("node",pid=81384,fd=19))       
tcp   LISTEN 0      4096         0.0.0.0:3306       0.0.0.0:*    users:(("docker-proxy",pid=64299,fd=4))
tcp   LISTEN 0      4096         0.0.0.0:8080       0.0.0.0:*    users:(("docker-proxy",pid=84393,fd=4))
tcp   LISTEN 0      4096            [::]:3000          [::]:*    users:(("docker-proxy",pid=64229,fd=4))
tcp   LISTEN 0      4096            [::]:3306          [::]:*    users:(("docker-proxy",pid=64306,fd=4))
tcp   LISTEN 0      4096            [::]:8080          [::]:*    users:(("docker-proxy",pid=84401,fd=4))

A short while later

C:\Users\paul.burridge>netstat -ano | find "LISTENING" | find "50"
  TCP    0.0.0.0:5040           0.0.0.0:0              LISTENING       10800
  TCP    127.0.0.1:5001         0.0.0.0:0              LISTENING       12160
PaulBurridge commented 1 year ago

wslhost.exe is grabbing any port opened in container for a short while even without vscode running.

Note that port does not appear to function while it exists.

WSL version: 1.1.3.0
Kernel version: 5.15.90.1
WSLg version: 1.0.49
MSRDC version: 1.2.3770
Direct3D version: 1.608.2-61064218
DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows version: 10.0.22000.1574
PaulBurridge commented 1 year ago

Issue with latest WSL version see https://github.com/microsoft/WSL/issues/9751

Workaround is to disable WSL2 port forwarding via localhostForwarding=false see https://learn.microsoft.com/en-us/windows/wsl/wsl-config for details how to configure.

chuanqisun commented 1 year ago

Same issue. Port number is always incremented by 1 from what I would like to use. I have also observed that

  1. The issue only happens with dev container. If I open the same folder in WSL, there is no port number error
  2. Inside the container, the port is not reported as being used.

image image

I don't have any workaround yet.

PaulBurridge commented 1 year ago

New bug raised for WSL https://github.com/microsoft/WSL/issues/9763

PaulBurridge commented 1 year ago

Pretty sure this is a WSL bug and there is not a lot that can be done by the VSCode developers.

WSL is attempting to forward a port from WSL OS to windows when it sees a new port opened in the container even if that port is not exposed to the WSL OS which will never work.

This means that the port is in use when VSCode extension tries to do its auto tunneling and has to use a different port which is probably the best behavior it can have in this scenario.

See my comment https://github.com/microsoft/vscode-remote-release/issues/8154#issuecomment-1460560114 which prevents this happening by disabling WSL's own port forwarding.

gund commented 1 year ago

Yes it is a WSL bug, specifically experimental.networkingMode=mirrored - when I set this in my .wslconfig file I start getting this behavior with incremented forwarded ports. But if I comment out that part in my config then everything works fine.

S-p-Ace commented 3 months ago

Looks like in the docs networkingMode is no longer listed as experimental. My problem now is that vscode appears to block the ports of my application(s) by forwarding them. (even if remote.autoForwardPorts is disabled)

netstat -ab shows that vscode ist listening on my application ports and launching my app results in an error that the ports are already in use. This does not happen when disabling networkingMode=mirrored in .wslconfig

Disabling localhostForwarding as suggested by PaulBurridge #8154 (comment) does not help in this case