Help: Port forwarding problem

[gerboland]

Hi guys, I'm combining vpnkit and hyperkit to boot ubuntu cloud images on my Mac with great success, but I'm hitting a wall with port forwarding. End-goal is to SSH into the VM from the host, but I'm stuck.

I'm doing the following (on OSX 10.12)

1. launching vpnkit:

vpnkit.exe --debug --ethernet=/tmp/vpnkit.socket --port=/tmp/vpnkit.port.socket --diagnostics /tmp/vpnkit.diag.socket --vsock-path=/tmp/connect --host-names mac.localhost

2. launch my VM:

./hyperkit -A -u -H -m 512M -c 1 -s 2:0,virtio-vpnkit,path=/tmp/vpnkit.socket \ -s "1:0,ahci-hd,file://$PWD/ubuntu-16.04-server-cloudimg-amd64-disk1.qcow2,format=qcow,qcow-config=discard=true;compact_after_unmaps=0;keep_erased=0;runtime_asserts=false" \ -s 0:0,hostbridge -s 31,lpc -l com1,stdio,log=log-ring \ -f kexec,$PWD/ubuntu-16.04-server-cloudimg-amd64-vmlinuz-generic,$PWD/ubuntu-16.04-server-cloudimg-amd64-initrd-generic,earlyprintk=serial\ console=ttyS0\ root=/dev/sda1\ rw -s 1:1,ahci-cd,/Users/gerry/VMs/ubuntu/image.iso \ -s 6,virtio-9p,path=/tmp/vpnkit.port.socket,tag=port -s 5,virtio-rnd -s 7,virtio-sock,guest_cid=3,path=/tmp,guest_forwards=8002

I get console access to a perfectly operational VM, which can access the network just fine.

Now I try to configure port forwarding, so inside the VM I do:

3. mount the 9p filesystem

sudo mount -t 9p -o trans=virtio,version=9p2000 port /port

4. try to forward port 8000 from the VM to the host

~/vpnkit-expose-port.linux -i -host-ip 127.0.0.1 -host-port 8000 -container-ip 0.0.0.0 -container-port 8000 -no-local-ip

5. launch a simple service:

python -m SimpleHTTPServer 8000

But on the host, trying to access that service hangs:

wget localhost:8000 --2017-07-14 15:22:10-- http://localhost:8000/ Resolving localhost... ::1, 127.0.0.1 Connecting to localhost|::1|:8000... connected. HTTP request sent, awaiting response...

Sometimes I see vpnkit print

vpnkit.exe: [ERROR] Socket.Stream: caught Socket is not connected

I've a couple of questions as a result:

1. Am I doing something wrong above? Cat-ing the diagnostics from vpnkit's diagnostics socket doesn't show me anything obviously wrong.
2. I see virtio-vsock support is implemented, which requires 4.8 kernels or later. But is port forwarding implemented using this protocol? (the guest_forwards= makes me suspicious)
3. is datakit required for this?
4. vpnkit doesn't need to be root for port forwarding, does it? I don't see how, but just in case.

Here are my logs in case they're useful: https://pastebin.ca/3842831 Any tips/tricks to debugging this would be greatly appreciated.

Many thanks in advance -G

[djs55]

Hi -- I think I can spot 2 problems:

• as you suspected, port forwarding is done over virtio-vsock. There are 2 parts to this: the control interface is currently a 9P filesystem mount manipulated by vpnkit-expose-port (although we'll probably replace this with GRPC or similar at some point). Once the port is "exposed" and vpnkit is listening on the host, incoming connections are forwarded over AF_VSOCK connections to a server process called vpnkit-forwarder -- I suspect this will fail to start if the kernel doesn't support the address family.

• secondly, I think there's a minor problem in the vpnkit-expose-port command line. When incoming connections are forwarded, vpnkit writes a short header telling vpnkit-forwarder the final destination to connect to which is the container-ip on the vpnkit-expose-port command line. So I think you to give it an IP bound inside the VM instead of 0.0.0.0:

~/vpnkit-expose-port.linux -i -host-ip 127.0.0.1 -host-port 8000 -container-ip 127.0.0.1 -container-port 8000 -no-local-ip

At one point in the past we did support using the internal network for port forwarding but unfortunately that code was removed :(

Another way you could set up access would be a reverse port forward with ssh -- inside the VM you should be able to ssh <vpnkit gateway IP> -R localhost:8000:127.0.0.1:8000 (if I've remembered the syntax properly). This will connect to the host's sshd and tell it to listen on localhost:8000 (on the host) and forward connections back to 127.0.0.1:8000 in the VM.

In case it's helpful, recent versions of vpnkit should respond to DNS queries for docker.for.mac.localhost with the vpnkit IP. (Although we'll probably have to change the DNS name used since we shouldn't have used localhost)

Hope this helps a little!

[gerboland]

Hey djs55, thank you so much for the info, you've cleared up a lot of for me. Reverse port forwarding sounds like a more flexible solution for my needs. Thanks again! -G