BastilleBSD / bastille

Bastille is an open-source system for automating deployment and management of containerized applications on FreeBSD.
https://bastillebsd.org
BSD 3-Clause "New" or "Revised" License
819 stars 129 forks source link

[ENHANCEMENT] Multiple interfaces for VNET jails #656

Open eyegog opened 8 months ago

eyegog commented 8 months ago

Is your feature request related to a problem? Please describe. I do have use cases where I would like to have jails with multiple interfaces attached to different bridges on different VLANs (mgmt, service, etc) for example:

mount.devfs;
exec.start="/bin/sh /etc/rc";
exec.stop="/bin/sh /etc/rc.shutdown";
devfs_ruleset="11";

myjail {
    path="/jails/myjail";
        host.hostname="$name.something.co.uk";
        enforce_statfs=2;
        vnet;
        vnet.interface+=myjail0_b;
        exec.prestart+="/usr/local/sbin/epair_create myjail0 bridge0";
        exec.poststop+="ifconfig myjail0_a destroy";
        vnet.interface+=myjail1_b;
        exec.prestart+="/usr/local/sbin/epair_create myjail1 bridge1";
        exec.poststop+="ifconfig myjail1_a destroy";

}

Inside jail:

$ ifconfig
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
    options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    inet 127.0.0.1 netmask 0xff000000
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0xd
    groups: lo
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0 metric 0 mtu 33152
    options=0
    groups: pflog
myjail0_b: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    options=8<VLAN_MTU>
    ether 02:c9:eb:88:6b:0b
    inet 10.10.99.14 netmask 0xffffff00 broadcast 10.10.99.255
    groups: epair
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
myjail1_b: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    options=8<VLAN_MTU>
    ether 02:4c:71:81:a1:0b
    inet 10.10.4.7 netmask 0xffffff00 broadcast 10.10.4.255
    groups: epair
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

As far as I am aware, Bastille doesn't support creating VNET jails with this kind of configuration.

Describe the solution you'd like For Bastille to be able to create jails with multiple interfaces, perhaps with something like the following syntax:

bastille create -B azkaban 13.2-RELEASE 192.168.1.50/24 bridge0 172.16.17.5/24 bridge1 

Describe alternatives you've considered As of the moment I've written other automation to create VNET jails with multiple interfaces but it would be nice to assimilate this functionality into a single tool.

Additional context Admittedly I am a newbie to the Bastille source code - but I am willing to contribute towards the development of this functionality and mainly I want to know if the dev team would be interested in such a feature :)

nreilly commented 8 months ago

This would be a great enhancement. I'm currently achieving the outcome by modifying the jail.conf by hand and using a patched jib to make it work nicely with multiple bridges.

--- jib 2023-12-21 21:05:09.900919000 +0000
+++ /usr/local/bin/jib  2023-08-25 01:16:54.224528000 +0000
@@ -259,17 +259,10 @@
    fi
 }

-jib_addm_usage="addm [-b BRIDGE_NAME] NAME [!]iface0 [[!]iface1 ...]"
+jib_addm_usage="addm NAME bridge0 bridge1 ...]"
 jib_addm_descr="Creates e0b_NAME [e1b_NAME ...]"
 jib_addm()
 {
-   local OPTIND=1 OPTARG flag bridge=bridge
-   while getopts b: flag; do
-       case "$flag" in
-       b) bridge="${OPTARG:-bridge}" ;;
-       *) action_usage addm # NOTREACHED
-       esac
-   done
    shift $(( $OPTIND - 1 ))

    local name="$1"

and then the relevant part of the jail.conf:

  vnet;
  vnet.interface = e0b_port, e1b_port, e2b_port, e3b_port, e4b_port, e5b_port, e6b_port, e7b_port, e8b_port;
  exec.prestart += "jib addm port a0 b0 c0 d0 a1 b1 c1 d1 a2";
  exec.poststop += "jib destroy port";

I also create the bridge interfaces via rc.conf, some bridged to local vlans.

ifconfig_genet0="up"
ifconfig_genet0_2="up"
ifconfig_genet0_4="up"
ifconfig_genet0_6="up"
ifconfig_genet0_8="up"
vlans_genet0="2 4 6 8"
cloned_interfaces="bridge0 bridge1 bridge2 bridge3 bridge4 bridge5 bridge6 bridge7 bridge8"
ifconfig_bridge0_name="a0"
ifconfig_bridge1_name="b0"
ifconfig_bridge2_name="c0"
ifconfig_bridge3_name="d0"
ifconfig_bridge4_name="a1"
ifconfig_bridge5_name="b1"
ifconfig_bridge6_name="c1"
ifconfig_bridge7_name="d1"
ifconfig_bridge8_name="a2"
ifconfig_a0="inet 10.15.0.1/24"
ifconfig_b0="inet 10.14.0.129/25"
ifconfig_c0="inet 10.14.0.1/25"
ifconfig_d0="inet 10.14.2.1/24"
autobridge_interfaces="a1 b1 c1 d1"
autobridge_a1="genet0.6"
autobridge_b1="genet0.2"
autobridge_c1="genet0.4"
autobridge_d1="genet0.8"
Notsonoble commented 5 months ago

I agree that this would be a good thing to make work. I'm considering moving from iocage to bastille based on the fact that iocage seems to be dead. However I need the ability to apply different vlans to different jails (or interfaces in the same jail) and public ip jails outside the subnet of the host.