mathiasp
commented
5 years ago I still don't understand the "permission denied", but I have it up and running. Hetzner routes the subnet as "behind" the main IP of the interface. e.g. my main interface has aa.aa.aa.aa and my subnet bb.bb.bb.bb/28 then from hetzner everything to the bb... subnet is routeted via my aa... address. The mac-addresses of the subnet are not visible from hetzner.
So, no bridging.
I just removed em0 from the switch public, gave it the first IP from my range, and set that as gateway in the vm. Voila, everything works.
But I'm losing one address for routing, and I need to use vimage for my jails that want a fixed address, if I understand this correctly. Not so nice :(
So in the host:
basis# vm switch info public
------------------------
Virtual Switch: public
------------------------
type: standard
ident: vm-public
vlan: -
physical-ports:
bytes-in: 3964259 (3.780M)
bytes-out: 15629472 (14.905M)
virtual-port
device: tap0
vm: mytest
basis# ifconfig vm-public
vm-public: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 36:54:87:28:e7:aa
inet 176.9.205.225 netmask 0xfffffff0 broadcast 176.9.205.239
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 4 priority 128 path cost 2000000
groups: bridge vm-switch viid-4c918@
nd6 options=1<PERFORMNUD>
basis# netstat -r
Routing tables
Internet:
Destination Gateway Flags Netif Expire
default static.1.52.63.178 UGS em0
localhost link#2 UH lo0
176.9.205.224/28 link#3 U vm-publi
static.225.205.9.1 link#3 UHS lo0And in the VM:
root@mailtest:~ # netstat -r
Routing tables
Internet:
Destination Gateway Flags Netif Expire
default static.225.205.9.1 UGS vtnet0
localhost link#2 UH lo0
176.9.205.224/28 link#1 U vtnet0
mailtest link#1 UHS lo0
Internet6:
Destination Gateway Flags Netif Expire
::/96 localhost UGRS lo0
localhost link#2 UH lo0
::ffff:0.0.0.0/96 localhost UGRS lo0
fe80::/10 localhost UGRS lo0
fe80::%lo0/64 link#2 U lo0
fe80::1%lo0 link#2 UHS lo0
ff02::/16 localhost UGRS lo0
root@mailtest:~ # fetch http://google.de/
fetch: http://google.de/: size of remote file is not known
fetch.out 11 kB 3501 kBps 00s
This is my first experiment with (vm-) bhyve. On a fresh 12.0-RELEASE-p3 server with pf still deactivated I set up vm-bhyve with switch public and interface em0 following the quickstart. Instalation of -disk1.iso works fine, network does not.
Since this is running at Hetzner, I have a single IP for my main interface plus a /28-subnet, which I want to use for the VMs, static configuration and no dhcp.
If I set the first address of the subnet on the bridge and use one ip from the subnet inside bhyve I can ping e.g. 8.8.8.8, but everything else (fetch, ssh, basically everything which uses another port I think) gives me "permission denied" - but I'm root???
I've never had this on bare metal, I do not even know where to start looking... Any ideas, help, debugging tips?
This is how it looks from the host:
This is how it looks from inside the VM:
Pinging works, but everything else give permission denied????