ioc start does not set IP address

urosgruber commented 5 years ago

As I mention before ioc start become broken when refactor was done. At least commit e903f88d4eeb6211771ae62f819024eba8ab66ff works fine. Here is a config

{
    "basejail": "yes",
    "id": "base",
    "ip4_addr": "lo1|172.16.1.1/32",
    "release": "11.2-RELEASE",
    "template": "no"
}

And output from start

hostid_strict_check is disabled
hostid_strict_check is disabled
[+] JailStart@ioc/base: OK [1.795s]
  [+] JailDependantsStart@ioc/base: No dependant jails [0.0s]
Configuring nameserver for Jail 'base'
  [+] JailResolverConfig@ioc/base: OK [0.003s]
resolv.conf copied from host
  [+] JailResourceLimitAction@ioc/base: disabled [0.0s]
  [+] JailHookPrestart@ioc/base: SKIPPED [0.0s]
  [+] BasejailStorageConfig@ioc/base: OK [0.047s]
Setting fstab auto-creation placeholder
fstab loaded from /iocage/jails/base/fstab
  [+] MountFstab@ioc/base: OK [0.0s]
  [+] JailAttach@ioc/base: OK [0.019s]
Reading devfs.rules from /etc/devfs.rules
  [+] JailHookCreated@ioc/base: SKIPPED [0.0s]
  [+] MountDevFS@ioc/base: OK [0.001s]
/etc/rc.conf was read from /iocage/jails/base/root/etc/rc.conf
Updated /etc/rc.conf data from /iocage/jails/base/root/etc/rc.conf
/etc/rc.conf was not modified - skipping write
  [+] JailHookStart@ioc/base: OK [1.477s]
Executing (interactive): /usr/sbin/jexec 164 /bin/sh -c /bin/sh /etc/rc
  ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib
  32-bit compatibility ldconfig path: /usr/lib32
  /etc/rc: WARNING: $hostname is not set -- see rc.conf(5).
  Creating and/or trimming log files.
  Starting syslogd.
  Clearing /tmp (X related).
  Updating motd:.
  sysctl: unknown oid 'net.inet.ip.fw.enable' at line 1

  Sat Apr  6 19:25:04 UTC 2019
  [+] JailHookPoststart@ioc/base: SKIPPED [0.001s]
base running as JID 164

Later ioc exec ioc/base ifconfig gives me this

lo1: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    groups: lo

Output from previous working version is

hostid_strict_check is disabled
hostid_strict_check is disabled
Configuring nameserver for Jail 'base'
[+] JailResolverConfig@ioc/base: OK [0.002s]
resolv.conf copied from host
[+] JailDependantsStart@ioc/base: No dependant jails [0.0s]
[-] JailLaunch@ioc/base: ...
/etc/rc.conf was read from /ioc/jails/base/root/etc/rc.conf
Updated /etc/rc.conf data from /ioc/jails/base/root/etc/rc.conf
[+] JailLaunch@ioc/base: OK [0.216s]
Setting fstab auto-creation placeholder
fstab loaded from /ioc/jails/base/fstab
Setting fstab auto-creation placeholder
fstab loaded from /ioc/jails/base/fstab
/ioc/jails/base/fstab written
Reading devfs.rules from /etc/devfs.rules
Executing (interactive): /usr/sbin/jail -c allow.mount.nullfs=0 allow.mount.fdescfs=0 allow.mount.zfs=0 allow.mount.procfs=0 allow.mount.devfs=0 allow.socket_af=0 allow.quotas=0 allow.chflags=0 allow.raw_sockets=0 allow.sysvipc=0 allow.set_hostname=1 ip6.saddrsel=1 ip6.addr=- ip4.saddrsel=1 ip4.addr=lo1|172.16.0.1/32 host.hostuuid=base host.domainname=local host.hostname=base children.max=0 devfs_ruleset=4 enforce_statfs=2 securelevel=2 path=/ioc/jails/base/root name=ioc-base exec.timeout=600 stop.timeout=30 exec.prestart="/ioc/jails/base/launch-scripts/prestart.sh" exec.jail_user=root mount.fstab=/ioc/jails/base/fstab mount.devfs=1 allow.dying persist exec.poststart="/ioc/jails/base/launch-scripts/poststart.sh"
  running exec.created hook on the host
  running exec.start hook in the jail
  ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib
  32-bit compatibility ldconfig path: /usr/lib32
  /etc/rc: WARNING: $hostname is not set -- see rc.conf(5).
  Creating and/or trimming log files.
  Starting syslogd.
  Clearing /tmp (X related).
  Updating motd:.
  sysctl: unknown oid 'net.inet.ip.fw.enable' at line 1

  Sat Apr  6 19:27:18 UTC 2019
  running exec.poststart hook on the host
Jail 'base' started with JID 138
base running as JID 138

gronke commented 5 years ago

Ah, well this explains my questions about the behavior removing IPs from a host after a jail was stopped. 🤔

With this hint, I finally understood what needs to be done here. Let me elaborate:

A IP address configured to a jail must not be taken by the host (currently it must be to hand it over to a jail)
The IP is set from within the jail right before the jail.created hook
The IP is removed from within the jail right after exec.stop, but before the jail is removed (jail_remove)
We cannot trust ifconfig within a jail, so there needs to be a check ensuring that the IP did never appear on the host interfaces

I'm glad that py-jail brought tight control over the start and stop process, because fitting the tasks precisely in between the known hooks would otherwise not be possible. Good timing @urosgruber 😏

gronke commented 5 years ago

Fixed in #710

bsdci / libioc

ioc start does not set IP address #706