Add a reference implementation of port forwarding

[bamurtaugh]

The specification includes port attributes, but the dev container CLI doesn't include port forwarding support.

[Chuxel]

👍On this one. While you can use appPort, this is currently only wired into the single container setup and is port publishing rather than full forwarding (e.g., if something only listens to localhost it won't work). While implementors can communicate with a server via text stream for devcontainer exec, it's pretty common to need to communicate on a port instead. There's then user scenarios as well where this is very important.

[MunsMan]

I would love this feature and I would try to make an implementation draft, but I have a couple of questions.

1. To my knowledge, Docker only supports port forwarding configurations by startup. I couldn't find any command/interface to publish ports after initial start up. I found some ways to automatically publish all ports via host (unsafe) or via bridge (I guess VS Code implementation). Likewise, I propose we need a way to disable all forwarding, for untrusted environments. So, I'm thinking about these options:

   1. 'Static forwarding' via the published port from docker
   2. 'Automatic forwarding' via a network bridge
   3. 'Static and automatic forwarding' by combining both options
   4. 'No forwarding' for security or personal preference.

2. GitHub Codespaces. If there is support for Codespaces planned, is forwarding to the actual host enough? Further forwarding should be done by external software, right?

[chrmarti]

I was thinking of dynamic port forwarding where we could start something along the lines of devcontainer port-forward --workspace-folder my_project and that would forward the ports configured in the devcontainer.json (plus any that we might list in command line arguments).

E.g., we have implemented that in the Dev Containers extension for VS Code by locally listening on ports and forward any connections using docker exec to run a little JavaScript snippet inside the container to connect to the target port there. The problem (and the reason we haven't included it in the dev container CLI) with this implementation is that it requires NodeJS inside the container. You could use socat inside the container to achieve the same, but that would then require socat to be available.

[MunsMan]

@chrmarti your example was a helpful confirmation, I had to reverse engineer VS Code to get that inside.

I added 'static' forwarding via devontainer.json in this branch (https://github.com/MunsMan/devcontainercli/tree/port_forwarding), there it implemented via the Docker CLI and the publish option. Currently, socat is not part of the default image to my knowledge, right? VS Code mounts it own node runtimes and server extensions, right?

For automatic forwarding, should it be a feature then? When we demand code in the container, it should be a feature. I was thinking if forwarding is possible by mounting sockets via volumes into the container, but I will build a prototype for myself first.

[MunsMan]

I built a prototype for automatic forwarding ports via a UNIX Socket (I had the assumption that WSL should support that), but even macOS didn't support sharing UNIX Sockets via a volume. The macOS support will not come in the near future (https://github.com/docker/for-mac/issues/483). I build the Prototype in Rust and build images for the corresponding container, which then can directly be mounted. For Linux to Container or Container to Container, I had a Response Time which was 50 Times faster than the one of VS Code.

Due to the missing support for Unix Sockets on the other platforms, I'm considering using one static Port and Multiplexing all the other ports through it. A small program needs to run in the Container, scan for new ports and forward all traffic though the port. I would build the program in Rust, and then the CLI would need to pull the container corresponding binary and mount it into the container. The execution can be started via the postCreateCommand. On the Host, an Application would Demultiplex the Traffic and manage the listening ports. Any Feedback on that approach?

[MunsMan]

I finished my prototype and when configured it works well for me. I published it in my autoForward repo. Currently, I have the problem, that I can't start a detached process on the host running, which just using initializeCommand, but maybe someone can give me a hind there. Furthermore, I have never packaged a feature, but I will hopefully figure that out.

It is pretty lightweight, especially compared to using VS Code for just auto-forwarding. Feedback is highly appreciated. Cheers

[davidkron]

@MunsMan Thank you for your efforts to actually implement this feature. This is the single feature that is preventing me from moving to devcontainers as a standard. Without this, using devcontainers without any specific tooling is simply impossible.

If I have some free time, I will try out your code.

[delijah]

Any progress on this? It would help us a lot! 🤩

[nohzafk]

For anyone interested in this topic, I have implemented the desired functionality using a Python script called devcontainer-cli-port-forwarder. This solution is based on @chrmarti's suggestion to utilize socat for port forwarding. To make it work, simply add the following two lines to your devcontainer.json file, and you should be all set.

"onCreateCommand": "sudo apt update && sudo apt install -y socat",
"initializeCommand": "python3 .devcontainer/devcontainer-cli-port-forwarder/forwarder.py &"

I was thinking of dynamic port forwarding where we could start something along the lines of devcontainer port-forward --workspace-folder my_project and that would forward the ports configured in the devcontainer.json (plus any that we might list in command line arguments).

E.g., we have implemented that in the Dev Containers extension for VS Code by locally listening on ports and forward any connections using docker exec to run a little JavaScript snippet inside the container to connect to the target port there. The problem (and the reason we haven't included it in the dev container CLI) with this implementation is that it requires NodeJS inside the container. You could use socat inside the container to achieve the same, but that would then require socat to be available.

[mikoto2000]

Until forwardPorts support is complete, how about treating forwardPorts as an alias for addPort?

It would require some workaround (e.g., add server process option --host 0.0.0.0), but I think we can achieve the connection to the container.

I think this behavior is effective when using an already existing devcontainer.json.

[beingminimal]

@chrmarti Can you please share ETA for this issue?