containers / docker-lvm-plugin

Docker volume plugin for LVM volumes
GNU Lesser General Public License v3.0
155 stars 64 forks source link

Fix mkfs.xfs cannot create volumes due to existing filesystems #62

Closed wborn closed 5 years ago

wborn commented 5 years ago

Whenever we delete and recreate a Rancher (v1.6.21) container stack often LVM volumes cannot be recreated because mkfs.xfs refuses to overwrite existing filesystems and logs an error like the following:

Nov 21 16:35:55 atomic docker-lvm-plugin[3191]: Create: mkfs.xfs error: exit status 1 output mkfs.xfs: /dev/atomicos/mystack_mosquitto_cf159 appears to contain an existing filesystem (xfs_external_log).
                                                mkfs.xfs: Use the -f option to force overwrite.

See also this full logging.txt.

As a result the affected containers fail to start because their volumes cannot be recreated.

This PR modifies the docker-lvm-plugin by adding the -f option to mkfs.xfs command so this issue no longer occurs.

Do you think it is a good idea to force overwrite by default or should we make this configurable with a configuration option?

Additional environment details:

maartenl945 commented 5 years ago

Yes, I've noticed the same problem that volumes can sometimes not be created with the plugin if they've been created and deleted before. Apparently, the delete doesn't fully clean-up the volume on the file system.

ashald commented 5 years ago

We ran into the same issue but we're somehow concerned with just forcing mkfs to overwrite the FS. It seems that the root cause of the issue is that lvcreate detects xfs signature at the beginning of the volume and asks for a confirmation to zero it (the default behavior). Because there is not stdin attached confirmation is not given and lvcreate just creates the volume without zeroing its beginning. This in turn leads to an error with mkfs. Adding -y to lvcreate forces a yes answer to zeriong question and makes sure that volume beginning will be properly cleared.

wborn commented 5 years ago

Thanks for merging the other fix @shishir-a412ed ! I'll close this PR now.