Container to Host networking does not work consistently

[davidebbo]

Context: this is based on discussions with @davidfowl on Discord.

When running on Docker Desktop, there is magic allowing a container to talk to the host, by using the special hostname host.docker.internal. However, this is not available on all environments. e.g. it likely doesn't work in the following cases:

1. Docker in WSL (see this StackOverflow question)
2. Docker engine on Windows without Docker Desktop
3. Podman
4. Docker Engine on Linux

This issue exists both for http and https traffic, but https presents an additional level of complexity. Even with Docker Desktop, it is problematic because containers don't have the self-signed cert that would allow them to talk to the host. ANd even if they did, it wouldn't work because the cert is only for localhost, and not host.docker.internal.

One potential solution is to use a reverse tunnel, similar to what @davidfowl does in https://github.com/davidfowl/YarpTunnelDemo. The trick that makes this work is that the communication is initiated from the Host to the proxy container, and not from a Container to the Host. And yet, its reverse nature effectively allows Container to Host communication, without ever needing to talk to a host port.

Here is an illustration (courtesy of @davidfowl):

[davidebbo]

It would be useful if Yarp had direct support for this, as tracked by https://github.com/microsoft/reverse-proxy/issues/1618.