[BUG] I get an error when creating a second jail using the create -B option.

[takkatou2006]

[MANDATORY] Describe the bug [MANDATORY] Error when creating second jail with create -B.

[MANDATORY] Bastille and FreeBSD version (paste bastille -v && freebsd-version -kru output)

0.9.20220216
13.0-RELEASE-p8
13.0-RELEASE-p8
13.0-RELEASE-p10

[MANDATORY] How did you install bastille? (port/pkg/git) pkg

[optional] Steps to reproduce?

# bastille -v
0.9.20220216
# bastille create -B jail1 13.0-RELEASE 192.168.0.10/24 bridge0
# bastille create -B jail2 13.0-RELEASE 192.168.0.11/24 bridge0
Valid: (192.168.0.11/24).
Valid: (bridge0).

Creating a thinjail...

[jail2]:
ifconfig: interface epair0 already exists
jail: jail2: ifconfig epair0 create: failed

[jail2]: Not started. See 'bastille start jail2'.
[jail2]: Not started. See 'bastille start jail2'.
[jail2]:
ifconfig: interface epair0 already exists
jail: jail2: ifconfig epair0 create: failed

# cat /usr/local/bastille/jails/jail2/jail.conf
jail2 {
  devfs_ruleset = 13;
  enforce_statfs = 2;
  exec.clean;
  exec.consolelog = /var/log/bastille/jail2_console.log;
  exec.start = '/bin/sh /etc/rc';
  exec.stop = '/bin/sh /etc/rc.shutdown';
  host.hostname = jail2;
  mount.devfs;
  mount.fstab = /usr/local/bastille/jails/jail2/fstab;
  path = /usr/local/bastille/jails/jail2/root;
  securelevel = 2;

  vnet;
  vnet.interface = "e0b_jail2";
  exec.prestart += "ifconfig epair0 create";
  exec.prestart += "ifconfig bridge0 addm epair0a";
  exec.prestart += "ifconfig epair0a up name e0a_jail2";
  exec.prestart += "ifconfig epair0b up name e0b_jail2";
  exec.poststop += "ifconfig bridge0 deletem e0a_jail2";
  exec.poststop += "ifconfig e0a_jail2 destroy";
}

[optional] Expected behavior A second jail is created with epair1.

[agrajag9]

Right here: https://github.com/BastilleBSD/bastille/blob/ff7de9167a6f03bd896f3a953c360ada1e4d85da/usr/local/share/bastille/common.sh#L76

That expression will never match what's being written to the config. The script writes out e${uniq_epair_bridge}b_${jail_name} but looks for e[0-9]b_bastille${_num}.

Of note, this will also break on systems with >10 jails.

[mmance]

I noticed this as well, I have to edit each jail I create after the first and increment the epair device.

[jfieber]

Baking in an epair number in the create also makes the strategy fundamentally incompatible with anything else on the system creating epair interfaces. It needs to start with capturing the output of ifconfig epair create and then working with the specific instance you got on this particular launch.

You also get a new random mac address each time you start the jail, which the jib managed epairs avoid.

[schrodyn]

I ran face first into this issue this week and effectively prevents me from using BastilleBSD. Without the ability to bridge jails into existing bridges all BastilleBSD is good for, for me, is managing the creation of the jails themselves (thin / thick) but I have to do all the manual post-configuration and setup myself.

[schrodyn]

Was wondering if there's any traction on fixing this issue? It's still preventing me from using Bastille.

[cedwards]

This has been resolved. I manually created three bridges and attached a new jail to each. No issues.

The original issue was caused by a regression in the previous release and has been fixed.

[joh-ku]

I still run into this issue after installing Bastille from origin/master on c760b81.

In my environment I've installed sysutils/iocage in parallel and I receive the following error when creating a new jail with Bastille (same as above):

root@freebsdtest:~ # bastille create -B bastille1 13.2-RELEASE 192.168.1.212/24 bridge0
Valid: (192.168.1.212/24).
Valid: (bridge0).

Creating a thinjail...

pfctl: /dev/pf: No such file or directory
[bastille1]:
ifconfig: interface epair0 already exists
jail: bastille1: ifconfig epair0 create: failed

[bastille1]: Not started. See 'bastille start bastille1'.
[bastille1]: Not started. See 'bastille start bastille1'.
pfctl: /dev/pf: No such file or directory
[bastille1]:
ifconfig: interface epair0 already exists
jail: bastille1: ifconfig epair0 create: failed

The error only occurs if at least one iocage jail is up and running.

The problem is, that Bastille does not check for already existing epair interfaces. It tries to infer the correct epair interface by looping over the number of jails returned from bastille list jails and a grep on "${bastille_jailsdir}"/*/jail.conf:

https://github.com/BastilleBSD/bastille/blob/c760b817bd4dbc1ac88d17940edb18c03f6b2199/usr/local/share/bastille/common.sh#L80-L96

By doing so, any epair interface not created through Bastille itself won't be recognized. In case an iocage jail is already up and running, epair0 exists and the jail startup command fails, as it tries to create epair0:

https://github.com/BastilleBSD/bastille/blob/c760b817bd4dbc1ac88d17940edb18c03f6b2199/usr/local/share/bastille/common.sh#L102

[jfieber]

No, this issue is not yet robustly resolved. Unless bastille is the only thing on the system creating epair interfaces, the current implementation is brittle. Handling the flexible epair creation directly in an exec.prestart is a bit painful. I think it would make more sense to fork and internalize jib as a bastille command that can flexibly handle both pre-existing and create-on-demand bridges.

I started a bit down that path, but the jib mechanism for providing stable mac addresses doesn't translate well unless you wire down the mac addresses on your externally created bridge. That is arguably not a problem bastille needs to solve, but if the existing jib behavior is internalized, an expectation that it be consistent between on-demand and pre-created bridges also isn't unreasonable.