VIDA-NYU / reprozip

ReproZip is a tool that simplifies the process of creating reproducible experiments from command-line executions, a frequently-used common denominator in computational science.
https://www.reprozip.org/
BSD 3-Clause "New" or "Revised" License
305 stars 34 forks source link

reprounzip error creating chroot #57

Closed gdevenyi closed 10 years ago

gdevenyi commented 10 years ago

Successfully packed up my test experiment and now I've moved it to another machine for testing, when I try and unpack a chroot:

ace@delta:~$ reprounzip chroot my_experiment.rpz test
Traceback (most recent call last):
  File "/usr/local/bin/reprounzip", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/reprounzip/reprounzip.py", line 52, in main
    args.func(args)
  File "/usr/local/lib/python2.7/dist-packages/reprounzip/unpackers/default.py", line 171, in create_chroot
    busybox_path)
  File "/usr/local/lib/python2.7/dist-packages/reprounzip/utils.py", line 145, in download_file
    cache.copy(dest)
  File "/usr/local/lib/python2.7/dist-packages/rpaths.py", line 824, in copy
    shutil.copy(self.path, self._to_backend(target))
  File "/usr/lib/python2.7/shutil.py", line 119, in copy
    copyfile(src, dst)
  File "/usr/lib/python2.7/shutil.py", line 83, in copyfile
    with open(dst, 'wb') as fdst:
IOError: [Errno 13] Permission denied: '/home/ace/test/root/bin/busybox'
remram44 commented 10 years ago

I think this "permission denied" means in fact that the root/bin directory doesn't exist. Should be fixed by 0d1f7aed.

Yet another code path that I do not cover, since my tests always use binaries from /bin/... ping #4

gdevenyi commented 10 years ago

Bug still exists, after upgrading python on the other system to solve that PATH bug,

ace@delta:~$ reprounzip chroot my_experiment.rpz test
Traceback (most recent call last):
  File "/usr/local/bin/reprounzip", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/reprounzip/reprounzip.py", line 58, in main
    args.func(args)
  File "/usr/local/lib/python2.7/dist-packages/reprounzip/unpackers/default.py", line 180, in create_chroot
    busybox_path)
  File "/usr/local/lib/python2.7/dist-packages/reprounzip/utils.py", line 147, in download_file
    cache.copy(dest)
  File "/usr/local/lib/python2.7/dist-packages/rpaths.py", line 824, in copy
    shutil.copy(self.path, self._to_backend(target))
  File "/usr/lib/python2.7/shutil.py", line 119, in copy
    copyfile(src, dst)
  File "/usr/lib/python2.7/shutil.py", line 83, in copyfile
    with open(dst, 'wb') as fdst:
IOError: [Errno 13] Permission denied: '/home/ace/test/root/bin/busybox'
remram44 commented 10 years ago

Oh, I've finally been able to reproduce your bug... The way I did it is by changing my /bin folder permissions to u-w (555 instead of 755; is that your setup?) and then unpacking as a regular user. reprounzip restores the same permissions on the directory and thus can't copy busybox into it. I'll have a fix shortly.

(probably affects directory, and the new "upload" commands on master as well)

gdevenyi commented 10 years ago

Is this bin on the machine I ran reprozip? Here's the details dr-xr-xr-x 2 root root 2120 Aug 1 13:08 bin

remram44 commented 10 years ago

Should be fixed by b99d5e54: reprounzip-0.3.2-3

Read-only directories are unusual, in fact even rm -rf should fail when you try to delete the unpacked directory (/home/ace/test), doesn't it?

gdevenyi commented 10 years ago

It does.

So if I understand the root cause, reprozip was creating directories, chmodding them, and then trying to copy into them, and failing to do so?

Perhaps chmod should move to the last step?

remram44 commented 10 years ago

Yes indeed this is what happened. I'm using the standard tarfile module to extract tarballs, and it restores the permissions after extracting the files, so I don't have a lot of control over this. I now make directories writable temporarily when copying into them (which only happens for busybox when unpacking a chroot as a non-root user).