search

simonvanderveldt / lxc-debian-wheezy-template

Since I don't use Debian nor LXC anymore this repo is abandoned. Feel free to fork it and continue :)
25 stars 13 forks source link

Fails with LVM Backed container option #1

Closed therealbill closed 5 years ago

therealbill commented 11 years ago

When using -B lvm, this template fails. Specifically, it fails when copying rootfs. It appears to unmount the rootfs from lvm somewhere and then proceed. Note: on the same system (Wheezy) it works if you do not specify a backing store.

Checking cache download in /var/cache/lxc/debian-wheezy/rootfs-amd64 ...                                                                         
Copying rootfs to /var/lib/lxc/wheezy-lvm/rootfs...mknod: `/var/lib/lxc/wheezy-lvm/rootfs/dev/tty1': No such file or directory                                                                      
mknod: `/var/lib/lxc/wheezy-lvm/rootfs/dev/tty2': No such file or directory                                                                         
mknod: `/var/lib/lxc/wheezy-lvm/rootfs/dev/tty3': No such file or directory                                                                           
mknod: `/var/lib/lxc/wheezy-lvm/rootfs/dev/tty4': No such file or directory                                                 
/usr/share/lxc/templates/lxc-debian-wheezy: line 42: /var/lib/lxc/wheezy-lvm/rootfs/etc/inittab: No such file or directory
simonvanderveldt commented 11 years ago

Thanks for reporting. Don't have LVM myself, so I never noticed it. As far as I can see from a quick look it isn't actually in the template since the rootfs location is determined from the path variable that lxc-create issues to the template. If lvm is used lxc-create mounts the lv to the rootfs path.

Is the lv actually created and mounted?

therealbill commented 11 years ago

Yes, it gets created and mounted, but for some reason when it it proceeds to copy to the rootfs it is unmounted somewhere. I figured it was a simple omission made during the change. I'll dig into it some more to assist finding it.

On Jun 9, 2013, at 10:49 AM, Simon van der Veldt notifications@github.com wrote:

Thanks for reporting. Don't have LVM myself, so I never noticed it. As far as I can see from a quick look it isn't actually in the template since the rootfs location is determined from the path variable that lxc-create issues to the template. If lvm is used lxc-create mounts the lv to the rootfs path.

Is the lv actually created and mounted?

— Reply to this email directly or view it on GitHub.

deimosfr commented 11 years ago

Hi,

I got the same issue and opened a bug report to Debian's LXC team.

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=716839

If you finded something, do not hesitate to report to the bug tracker.

Thanks

rbrockway commented 11 years ago

I worked out a fix and posted it to the Debian bug. Note - this breaks lxc-create with non-LVM. It is late here so I'll see if I have time to revisit this tomorrow.

The problem turned out to be that when the line "cp -a $cache/rootfs-$arch $rootfs || return 1" is executed in /usr/share/lxc/templates/lxc-debian-wheezy it behaves differently because $rootfs is a mount point. As a result it copies rootfs-$arch to be a subdirecctory of $rootfs. When file copies are subsequently attempted the tree is wrong and the install fails.

Here is a unified diff which makes this work with LVM:

 --- lxc-debian-wheezy   2013-07-21 01:49:25.518395282 +1000
 +++ /usr/share/lxc/templates/lxc-debian-wheezy  2013-07-21
 01:49:35.721923899 +1000
 @@ -125,7 +125,7 @@

      # make a local copy of the minidebian
      echo -n "Copying rootfs to $rootfs..."
 -    cp -a $cache/rootfs-$arch $rootfs || return 1 
 +    cp -a $cache/rootfs-$arch/* $rootfs || return 1
      return 0
  }

I had to hand adjust it after pasting so I hope it is ok. In any case it is a simple change.

therealbill commented 11 years ago

Good catch. An easy way to have both work would be either use rsync which should handle both, or use a conditional such as (pseudo code) "if $rootfs is mounted:".

On Jul 20, 2013, at 11:15 AM, rbrockway notifications@github.com wrote:

I worked out a fix and posted it to the Debian bug. Note - this breaks lxc-create with non-LVM. It is late here so I'll see if I have time to revisit this tomorrow.

The problem turned out to be that when the line "cp -a $cache/rootfs-$arch $rootfs || return 1" is executed in /usr/share/lxc/templates/lxc-debian-wheezy it behaves differently because $rootfs is a mount point. As a result it copies rootfs-$arch to be a subdirecctory of $rootfs. When file copies are subsequently attempted the tree is wrong and the install fails.

Here is a unified diff which makes this work with LVM:

--- lxc-debian-wheezy 2013-07-21 01:49:25.518395282 +1000 +++ /usr/share/lxc/templates/lxc-debian-wheezy 2013-07-21 01:49:35.721923899 +1000 @@ -125,7 +125,7 @@

make a local copy of the minidebian

echo -n "Copying rootfs to $rootfs..." cp -a $cache/rootfs-$arch $rootfs || return 1 cp -a $cache/rootfs-$arch/* $rootfs || return 1 return 0 } — Reply to this email directly or view it on GitHub.

simonvanderveldt commented 11 years ago

Nice find! Is it normal/expected behaviour that cp-ing to a mount point results in the files being created in a sub-directory? Never seen that before. Or is it LVM specific?

Are you sure cp -a $cache/rootfs-$arch/* $rootfs doesn't work without LVM? What do you guys think is the cleanest fix, an if (based on $backingstore maybe?) or rsyncing?

therealbill commented 11 years ago

Copying files and directories is a filesystem level event, so being LVM or not is not a factor.

What it probably is related to is differences in the cp command based on the trailing slash. For example:

cp /tmp/foo/ /tmp/bar vs cp /tmp/foo /tmp/bar

The latter says "copy the directory" and former "copy the contents of the directory"

Where it can bite you unknowingly is whether the destination directory exists first.

cp -a /tmp/foo /tmp/bar if /tmp/bar does not exist /tmp/bar is created then the contents of /tmp/foo are copied into it. If /tmp/bar exists, the th directory /tmp/foo is copied into /tmp/bar -> /tmp/foo/bar/{contents of /tmp/foo}

Contrast this to cp -a /tmp/foo/ /tmp/bar Whether /tmp/bar exists or not, the result is the same: /tmp/bar contains the contents of /tmp/foo.

So the problem could be fixable by using the trailing slash to cover both scenarios. Perhaps try ensuring the cp -a /$cache/rootfs-$arch actually ends with a "/". That should cover both situations. I'm not at work so I don't have my test environments to try it, but that should cover it nicely.

Hope that helps, Bill

On Jul 20, 2013, at 12:27 PM, Simon van der Veldt wrote:

Nice find! Is it normal/expected behaviour that cp-ing to a mount point results in the files being created in a sub-directory? Never experienced it before. Or is it LVM specific?

Are you sure the cp -a $cache/rootfs-$arch/* $rootfs doesn't work without LVM? In that case I think rsyncing is the cleanest option. What do you guys think?

— Reply to this email directly or view it on GitHub.

rbrockway commented 11 years ago

@simonvanderveldt -I was imprecise sorry. What I meant was that the cp -a behaved differently because $rootfs existed and was a directory. It happened to exist and be a directory because it was needed for the mount point. This is what I meant originally but I didn't explain it very well.

@TheRealBill - I tried the trailing / and it didn't provide the desired behaviour. I didn't expect to see a change in behaviour just by using a trailing / and quick tests I ran suggest it doesn't make a difference.

I'm going to use [ -d ] to test if $rootfs exists and is a directory and change the cp -a based on this result.

rbrockway commented 11 years ago

Ok this unified diff works in both with the default backingstore and LVM. I haven't tested the other options but they should also work. FWIW the debian package maintainer rejected my suggested fix and instead recommends upgrading with tar.

  --- lxc-debian-wheezy   2013-07-21 01:49:25.518395282 +1000
  +++ /usr/share/lxc/templates/lxc-debian-wheezy  2013-07-21 11:56:48.479221496 +1000
  @@ -124,8 +124,16 @@
       rootfs=$3

       # make a local copy of the minidebian
  +       # We need to behave differently if $rootfs exists and is a
  +       # directory, as happens when we use LVM as a backing store.
       echo -n "Copying rootfs to $rootfs..."
  -    cp -a $cache/rootfs-$arch $rootfs || return 1
  +       if [ -d $rootfs ]
  +       then
  +       cp -a $cache/rootfs-$arch/* $rootfs || return 1
  +       else
  +       cp -a $cache/rootfs-$arch $rootfs || return 1
  +       fi
  +
       return 0
   }