search

bsdci / libioc

A Python library to manage jails with ioc{age,ell}
https://bsd.ci/libioc
Other
38 stars 11 forks source link

after stop, jail's ZFS device(s) still busy #622

Closed igalic closed 5 years ago

igalic commented 5 years ago 
root@container-host1 ~# ioc stop -f quassel
[+] JailDestroy@quassel: SKIPPED [0.113s]
[55/1899]
Destroying jail quassel failed
quassel stopped
root@container-host1 ~# ioc destroy -f quassel
[-] ZFSDatasetDestroy@zroot/iocage/jails/quassel: ...
Traceback (most recent call last):
  File "/usr/local/bin/ioc", line 10, in <module>
    sys.dd:exit(cli())
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/src/ioc/ioc_cli/destroy.py", line 126, in cli
    ctx.parent.print_events(item.destroy())
  File "/usr/local/src/ioc/ioc_cli/__init__.py", line 87, in print_events
    for event in generator:
  File "/usr/local/lib/python3.6/site-packages/libioc/Jail.py", line 1164, in destroy
    raise e
  File "/usr/local/lib/python3.6/site-packages/libioc/Jail.py", line 1161, in destroy
    self.zfs.delete_dataset_recursive(self.dataset)
  File "/usr/local/lib/python3.6/site-packages/libioc/ZFS.py", line 96, in delete_dataset_recursive
    self.delete_dataset_recursive(child)
  File "/usr/local/lib/python3.6/site-packages/libioc/ZFS.py", line 101, in delete_dataset_recursive
    dataset.umount()
  File "libzfs.pyx", line 2547, in libzfs.ZFSDataset.umount
libzfs.ZFSException: Device busy
root@container-host1 ~#
igalic commented 5 years ago

figured out why the device busy thing is coming:

root@container-host ~# mount | grep quassel
zroot/ioc/jails/quassel on /iocage/jails/quassel (zfs, local, noatime, nfsv4acls)
zroot/ioc/jails/quassel/root on /iocage/jails/quassel/root (zfs, local, noatime, nfsv4acls)
/iocage/pkg/12/cache on /iocage/jails/quassel/root/.ioc-pkg (nullfs, local, read-only)
/usr/ports on /iocage/jails/quassel/root/usr/ports (nullfs, local)
/usr/obj on /iocage/jails/quassel/root/usr/obj (nullfs, local)

because there's still a bunch of things mounted after stop, but our destroy assumes it's clean.

gronke commented 5 years ago

@igalic Have you mounted /usr/ports and /usr/obj from fstab?

/iocage/jails/quassel/root/.ioc-pkg should be automatically unmounted. Will write a test for it.

igalic commented 5 years ago

Have you mounted /usr/ports and /usr/obj from fstab?

yes, i have.

igalic commented 5 years ago

btw, this also affects

root@webserver ~# ioc -d debug provision webproxy
[-] JailProvisioning@webproxy: ...
  [+] JailProvisioningAssetDownload@webproxy: OK [0.001s]
  [+] PackageConfiguration@webproxy: OK [0.003s]
Safely creating /iocage/jails/webproxy/root//usr/local/etc/pkg/repos directory
  [+] PackageInstall@webproxy: OK [3.846s]
Adding line to fstab: /root     /iocage/jails/webproxy/root/root        nullfs  ro      0       0
Adding line to fstab: /iocage/pkg/12/cache      /iocage/jails/webproxy/root/.ioc-pkg    nullfs  ro      0       0
fstab loaded from /iocage/jails/webproxy/fstab
Replacing fstab line with destination /iocage/jails/webproxy/root/.ioc-pkg
Deleting fstab entry: /iocage/pkg/12/cache -> /iocage/jails/webproxy/root/.ioc-pkg
Unmounting /iocage/jails/webproxy/root/.ioc-pkg
Jail mountpoint /iocage/jails/webproxy/root/.ioc-pkg umounted
auto-mount /iocage/jails/webproxy/root/.ioc-pkg
/iocage/pkg/12/cache mounted to running jail webproxy
/iocage/jails/webproxy/fstab written
Adding line to fstab: /usr/local/etc/puppet/environments        /iocage/jails/webproxy/root/usr/local/etc/puppet/environments   nullfs  ro      0       0
auto-mount /iocage/jails/webproxy/root/usr/local/etc/puppet/environments
Command exited with 1: /sbin/mount -o ro -t nullfs /usr/local/etc/puppet/environments /iocage/jails/webproxy/root/usr/local/etc/puppet/environments
    mount_nullfs: /iocage/jails/webproxy/root/usr/local/etc/puppet/environments: Resource deadlock avoided
Command exited with 1