search

BastilleBSD / bastille

Bastille is an open-source system for automating deployment and management of containerized applications on FreeBSD.
https://bastillebsd.org
BSD 3-Clause "New" or "Revised" License
854 stars 136 forks source link

[BUG] Jail to jail communication with two nodes #545

Open ivomarino opened 2 years ago

ivomarino commented 2 years ago

[MANDATORY] Describe the bug [MANDATORY] A clear and concise description of what the bug is.

[MANDATORY] Bastille and FreeBSD version (paste bastille -v && freebsd-version -kru output)

sudo bastille -v && freebsd-version -kru
0.9.20220216
13.1-RELEASE-p2
13.1-RELEASE-p2
13.1-RELEASE-p2

[MANDATORY] How did you install bastille? (port/pkg/git)

pkg

Hello, I'm in the process of migrating a bigger hosting site from Ubuntu to FreeBSD and the hoster where I need to migrate all this runs OpenStack. From there I get two FreeBSD VMs, bsd-01 and bsd-02, I'm running bastille on both nodes in order to setup jailed environments.

I create compute-01 and compute-02 jail on bsd-01 and compute-03 on bsd-03. The two BSD nodes can see (ping) each other but the jails hosted on bsd-01 can't see or ping the jails on bsd-02 (yes, raw sockets has been enabled, bsd-01 can ping it's jails). The two bsd-01 and 02 VMs have virtual vtnetX interface vtnet0, I tried to create jails on bsd-01 using loopback, shared ip and vnet.

I noticed the following boradcast setting:

~ ifconfig -n | grep 172
inet 172.31.10.10 netmask 0xfffff000 broadcast 172.31.15.255  # VM bsd-01
inet 172.31.10.35 netmask 0xffffffff broadcast 172.31.10.35   # Jail compute-01

172.31.10.10 is bsd-01 IP (VM) and 172.31.10.35 is the IP of the jail using aliasing on vtnet0, the broadcast seems different, could this be a potential issue for network communication between jails running on bsd-01 and jails running on bsd-02?

Thanks in advance on any feedback.

adriel-tech commented 2 years ago

I had a similar issue caused by improper broadcast, you can edit the jails jail.conf and add /24 to the IP address. ip4.addr = 172.31.10.35/24;. When creating a jail you need to add the /24 or it will default to a subnet as yours did.

I can't really picture what you are doing in my brain. You are using loopback and vnet? Make sure your firewall settings are not causing issues with the vnet interface on the host.

bmac2 commented 1 year ago

@ivomarino was the solution said above sufficient? Is this still an issue?