search

andsens / bootstrap-vz

Bootstrap Debian images for virtualized environments
http://bootstrap-vz.readthedocs.io/
Other
263 stars 145 forks source link

AttributeError: 'Mount' object has no attribute 'mount_dir' #497

Open dereckson opened 5 years ago

dereckson commented 5 years ago

For an unidentified reason, it wasn't possible to unmount the bootstrap volume (lsof / fuser didn't show anything using the disk).

In such scenario, the code logic tries to read a non existing mount_dir attribute. The issue was reproductible.

Tested with current master code.

Here a log from our CI/CD environment to with the relevant stacktrace.

error   24-Apr-2019 16:09:27    Unmounting the bootstrap volume
error   24-Apr-2019 16:09:27    umount: /target/4a784653/root/dev: target is busy
error   24-Apr-2019 16:09:27    (In some cases useful info about processes that
error   24-Apr-2019 16:09:27    use the device is found by lsof(8) or fuser(1).)
error   24-Apr-2019 16:09:28    Command 'umount /target/4a784653/root/dev' returned non-zero exit status 32
error   24-Apr-2019 16:09:28    Traceback (most recent call last):
error   24-Apr-2019 16:09:28      File "/bootstrap/debian-template-593/bootstrap/bootstrapvz/base/main.py", line 111, in run
error   24-Apr-2019 16:09:28        tasklist.run(info=bootstrap_info, dry_run=dry_run)
error   24-Apr-2019 16:09:28      File "/bootstrap/debian-template-593/bootstrap/bootstrapvz/base/tasklist.py", line 44, in run
error   24-Apr-2019 16:09:28        task.run(info)
error   24-Apr-2019 16:09:28      File "/bootstrap/debian-template-593/bootstrap/bootstrapvz/common/tasks/filesystem.py", line 189, in run
error   24-Apr-2019 16:09:28        info.volume.partition_map.root.unmount()
error   24-Apr-2019 16:09:28      File "/bootstrap/debian-template-593/bootstrap/bootstrapvz/common/fsm_proxy.py", line 39, in proxy
error   24-Apr-2019 16:09:28        fn(**kwargs)
error   24-Apr-2019 16:09:28      File "/usr/lib/python2.7/dist-packages/fysom/__init__.py", line 283, in fn
error   24-Apr-2019 16:09:28        if self._before_event(e) is False:
error   24-Apr-2019 16:09:28      File "/usr/lib/python2.7/dist-packages/fysom/__init__.py", line 318, in _before_event
error   24-Apr-2019 16:09:28        return getattr(self, fnname)(e)
error   24-Apr-2019 16:09:28      File "/usr/lib/python2.7/dist-packages/fysom/__init__.py", line 89, in _callback
error   24-Apr-2019 16:09:28        return func(obj, *args, **kwargs)
error   24-Apr-2019 16:09:28      File "/bootstrap/debian-template-593/bootstrap/bootstrapvz/base/fs/partitions/abstract.py", line 107, in _before_unmount
error   24-Apr-2019 16:09:28        self.mounts[destination].unmount()
error   24-Apr-2019 16:09:28      File "/bootstrap/debian-template-593/bootstrap/bootstrapvz/base/fs/partitions/mount.py", line 39, in unmount
error   24-Apr-2019 16:09:28        log_check_call(['umount', self.mount_dir])
error   24-Apr-2019 16:09:28      File "/bootstrap/debian-template-593/bootstrap/bootstrapvz/common/tools.py", line 14, in log_check_call
error   24-Apr-2019 16:09:28        raise e
error   24-Apr-2019 16:09:28    CalledProcessError: Command 'umount /target/4a784653/root/dev' returned non-zero exit status 32
error   24-Apr-2019 16:09:28    Rolling back
error   24-Apr-2019 16:09:31    Unmounting the bootstrap volume
error   24-Apr-2019 16:09:31    Traceback (most recent call last):
error   24-Apr-2019 16:09:31      File "bootstrap-vz", line 5, in <module>
error   24-Apr-2019 16:09:31        main()
error   24-Apr-2019 16:09:31      File "/bootstrap/debian-template-593/bootstrap/bootstrapvz/base/main.py", line 29, in main
error   24-Apr-2019 16:09:31        dry_run=opts['--dry-run'])
error   24-Apr-2019 16:09:31      File "/bootstrap/debian-template-593/bootstrap/bootstrapvz/base/main.py", line 141, in run
error   24-Apr-2019 16:09:31        rollback_tasklist.run(info=bootstrap_info, dry_run=dry_run)
error   24-Apr-2019 16:09:31      File "/bootstrap/debian-template-593/bootstrap/bootstrapvz/base/tasklist.py", line 44, in run
error   24-Apr-2019 16:09:31        task.run(info)
error   24-Apr-2019 16:09:31      File "/bootstrap/debian-template-593/bootstrap/bootstrapvz/common/tasks/filesystem.py", line 189, in run
error   24-Apr-2019 16:09:31        info.volume.partition_map.root.unmount()
error   24-Apr-2019 16:09:31      File "/bootstrap/debian-template-593/bootstrap/bootstrapvz/common/fsm_proxy.py", line 39, in proxy
error   24-Apr-2019 16:09:31        fn(**kwargs)
error   24-Apr-2019 16:09:31      File "/usr/lib/python2.7/dist-packages/fysom/__init__.py", line 283, in fn
error   24-Apr-2019 16:09:31        if self._before_event(e) is False:
error   24-Apr-2019 16:09:31      File "/usr/lib/python2.7/dist-packages/fysom/__init__.py", line 318, in _before_event
error   24-Apr-2019 16:09:31        return getattr(self, fnname)(e)
error   24-Apr-2019 16:09:31      File "/usr/lib/python2.7/dist-packages/fysom/__init__.py", line 89, in _callback
error   24-Apr-2019 16:09:31        return func(obj, *args, **kwargs)
error   24-Apr-2019 16:09:31      File "/bootstrap/debian-template-593/bootstrap/bootstrapvz/base/fs/partitions/abstract.py", line 107, in _before_unmount
error   24-Apr-2019 16:09:31        self.mounts[destination].unmount()
error   24-Apr-2019 16:09:31      File "/bootstrap/debian-template-593/bootstrap/bootstrapvz/base/fs/partitions/mount.py", line 39, in unmount
error   24-Apr-2019 16:09:31        log_check_call(['umount', self.mount_dir])
error   24-Apr-2019 16:09:31    AttributeError: 'Mount' object has no attribute 'mount_dir'
andsens commented 5 years ago

Please post the full log + manifest for this run. Something is not adding up. How can self.mount_dir not exist on the exact same object where it existed just a few milliseconds ago?
The relevant lines are here: https://github.com/andsens/bootstrap-vz/blob/32e8bbcf49114a675d721698549265f986be1dcd/bootstrapvz/base/fs/partitions/mount.py#L38-L40

del self.mount_dir is only run when the unmount is successful and no other part of the code removes that attribute.