hashicorp / nomad-driver-virt

Mozilla Public License 2.0
8 stars 1 forks source link

net: Nomad service port forwarding from client host machine local to VM #48

Open jrasell opened 1 month ago

jrasell commented 1 month ago

The current iptables rules used, as per libvirts forwarding incoming connections doc, do not allow for connections coming from the local Nomad client host to VM's running on it via the client interface IP address.

root@libvirt-0:/nomad-test# nomad service info jrasell-workstation-ssh
Job ID      Address          Tags  Node ID   Alloc ID
jrasell-vm  10.0.1.17:22982  []    a400db6e  6d0386a6
root@libvirt-0:/nomad-test# ssh 10.0.1.17 -p 22982
ssh: connect to host 10.0.1.17 port 22982: Connection refused

The virsh console command and using the libvirt DHCP assigned address work as expected and can be used as a workaround until this issue is fixed.

root@libvirt-0:/nomad-test# virsh console vm-795c791f
Connected to domain 'vm-795c791f'
Escape character is ^] (Ctrl + ])
root@libvirt-0:/nomad-test# virsh net-dhcp-leases default
 Expiry Time           MAC address         Protocol   IP address          Hostname            Client ID or DUID
--------------------------------------------------------------------------------------------------------------------------------------------------------
 2024-09-27 10:14:20   52:54:00:0d:a6:7b   ipv4       192.168.122.10/24   nomad-vm-795c791f   ff:08:24:45:0e:00:02:00:00:ab:11:d5:ae:0a:38:57:5f:a4:96

root@libvirt-0:/nomad-test# ssh ubuntu@192.168.122.10
Welcome to Ubuntu 20.04.6 LTS (GNU/Linux 5.4.0-193-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/pro

 System information as of Fri Sep 27 09:18:42 UTC 2024

  System load:  0.02              Processes:               125
  Usage of /:   70.5% of 1.96GB   Users logged in:         0
  Memory usage: 0%                IPv4 address for enp0s5: 192.168.122.10
  Swap usage:   0%

Expanded Security Maintenance for Applications is not enabled.

0 updates can be applied immediately.

Enable ESM Apps to receive additional future security updates.
See https://ubuntu.com/esm or run: sudo pro status

The list of available updates is more than a week old.
To check for new updates run: sudo apt update

To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

Kernel logs showing iptables logging entries for attempting the local client ssh connection:

2024-09-27T09:31:05.852474+00:00 libvirt-0 kernel: IN=lo OUT= MAC=00:00:00:00:00:00:00:00:00:00:00:00:08:00 SRC=10.0.1.17 DST=10.0.1.17 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=48746 DF PROTO=TCP SPT=59862 DPT=22982 WINDOW=33280 RES=0x00 SYN URGP=0
2024-09-27T09:31:05.852486+00:00 libvirt-0 kernel: IN=lo OUT= MAC=00:00:00:00:00:00:00:00:00:00:00:00:08:00 SRC=10.0.1.17 DST=10.0.1.17 LEN=40 TOS=0x10 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=22982 DPT=59862 WINDOW=0 RES=0x00 ACK RST URGP=0
2024-09-27T09:31:06.896442+00:00 libvirt-0 kernel: IN=lo OUT= MAC=00:00:00:00:00:00:00:00:00:00:00:00:08:00 SRC=10.0.1.17 DST=10.0.1.17 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=35825 DF PROTO=TCP SPT=59866 DPT=22982 WINDOW=33280 RES=0x00 SYN URGP=0
2024-09-27T09:31:06.896454+00:00 libvirt-0 kernel: IN=lo OUT= MAC=00:00:00:00:00:00:00:00:00:00:00:00:08:00 SRC=10.0.1.17 DST=10.0.1.17 LEN=40 TOS=0x10 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=22982 DPT=59866 WINDOW=0 RES=0x00 ACK RST URGP=0
2024-09-27T09:32:35.909451+00:00 libvirt-0 kernel: IN=lo OUT= MAC=00:00:00:00:00:00:00:00:00:00:00:00:08:00 SRC=10.0.1.17 DST=10.0.1.17 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=46814 DF PROTO=TCP SPT=53884 DPT=22982 WINDOW=33280 RES=0x00 SYN URGP=0
2024-09-27T09:32:35.909462+00:00 libvirt-0 kernel: IN=lo OUT= MAC=00:00:00:00:00:00:00:00:00:00:00:00:08:00 SRC=10.0.1.17 DST=10.0.1.17 LEN=40 TOS=0x10 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=22982 DPT=53884 WINDOW=0 RES=0x00 ACK RST URGP=0