delphix / appliance-build

This repository contains the code used to build the Ubuntu-based Delphix Appliance, leveraging open-source tools such as Debian's live-build, Docker, Ansible, OpenZFS, and others.
Apache License 2.0
19 stars 41 forks source link

TOOL-11972 appliance-build on focal: cannot export 'rpool': pool is busy #698

Closed sebroy closed 1 year ago

sebroy commented 1 year ago

Problem

Exporting the rpool that is being installed by appliance-build sometimes fails with EBUSY.

Solution

While we don't have a root-cause for the EBUSY failure, this fix attempts to workaround it. The umount step that is done immediately before the export also previously failed with EBUSY, but a retry loop was added to that step which proved to be a viable workaround. The same approach is attempted here with the export step.

Testing Done

ab-pre-push: http://selfservice.jenkins.delphix.com/job/appliance-build-orchestrator-pre-push/3641/

sdimitro commented 1 year ago

I'm ok with adding this to the code but from the last time that I looked into this it seemed like a retry loop wouldn't help. I think the issue that there is some refcount bug in the zfs module used (an old version of Canonical's ZFS) and we'd leak a refcount so regardless of how much you wait or how many times you would call zpool export -f you'd always get EBUSY.