setup.py might be called from the wrong directory

[remram44]

I always assumed that noone in their right mind would call a setup.py script from another directory. pip does. I had to do this change:

diff --git a/reprozip/setup.py b/reprozip/setup.py
index 67c6ab5..fd4597b 100644
--- a/reprozip/setup.py
+++ b/reprozip/setup.py
@@ -6,3 +6,7 @@ from setuptools import setup, Extension
+# pip workaround
+os.chdir(os.path.abspath(os.path.dirname(__file__)))
+
+
 # List the source files
 sources = ['pytracer.c', 'tracer.c', 'syscalls.c', 'database.c',
            'ptrace_utils.c', 'utils.c', 'log.c']

for this command to work:

$ pip install -e 'git+https://github.com/ViDA-NYU/reprozip.git@master#egg=reprozip&subdirectory=reprozip'

(also, why doesn't subdirectory= work without -e?)

Looks like a bug to me.

[remram44]

Also, afterwards, I still can't run my tool:

Traceback (most recent call last):
  File "/home/vagrant/venv/bin/reprozip", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/distribute-0.6.24-py2.7.egg/pkg_resources.py", line 2707, in <module>
    working_set.require(__requires__)
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/distribute-0.6.24-py2.7.egg/pkg_resources.py", line 686, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/distribute-0.6.24-py2.7.egg/pkg_resources.py", line 584, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: rpaths>=0.7

(although rpaths==0.8 was installed) full transcript

Removing rpaths>=0.7 from $MYVENV/src/reprozip/reprozip/reprozip.egg-info/requires.txt allows it to run. This might not be a pip issue (setuptools, I'm looking at you).

[Ivoz]

I'd try deleting / uninstalling distribute and installing a recent version of setuptools

[remram44]

remram@vebian:/tmp$ virtualenv venv
Running virtualenv with interpreter /usr/bin/python2
New python executable in venv/bin/python2
Not overwriting existing python script venv/bin/python (you must use venv/bin/python2)
Installing setuptools, pip...done.
remram@vebian:/tmp$ . venv/bin/activate
(venv)remram@vebian:/tmp$ pip install -U setuptools
Downloading/unpacking setuptools from https://pypi.python.org/packages/3.4/s/setuptools/setuptools-5.7-py2.py3-none-any.whl#md5=94eedce8d9b793d4affa4a85cd45edfa
  Downloading setuptools-5.7-py2.py3-none-any.whl (528kB): 528kB downloaded
Installing collected packages: setuptools
  Found existing installation: setuptools 5.4.1
    Uninstalling setuptools:
      Successfully uninstalled setuptools
Successfully installed setuptools
Cleaning up...
(venv)remram@vebian:/tmp$ pip install -U pip
Requirement already up-to-date: pip in ./venv/lib/python2.7/site-packages
Cleaning up...
(venv)remram@vebian:/tmp$ pip --version
pip 1.5.6 from /tmp/venv/local/lib/python2.7/site-packages (python 2.7)
(venv)remram@vebian:/tmp$ python
Python 2.7.8 (default, Jul  4 2014, 13:08:34) 
[GCC 4.9.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import distribute
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named distribute
>>> import setuptools
>>> setuptools.__version__
'5.7'
>>> 
(venv)remram@vebian:/tmp$ pip freeze
argparse==1.2.1
wsgiref==0.1.2
(venv)remram@vebian:/tmp$ pip install -e 'git+https://github.com/ViDA-NYU/reprozip.git@c6a0a83706d1065002171de9f2fbd63b4f63f751#egg=reprozip&subdirectory=reprozip'
Obtaining reprozip from git+https://github.com/ViDA-NYU/reprozip.git@c6a0a83706d1065002171de9f2fbd63b4f63f751#egg=reprozip&subdirectory=reprozip
  Cloning https://github.com/ViDA-NYU/reprozip.git (to c6a0a83706d1065002171de9f2fbd63b4f63f751) to ./venv/src/reprozip
  Could not find a tag or branch 'c6a0a83706d1065002171de9f2fbd63b4f63f751', assuming commit.
  Running setup.py (path:/tmp/venv/src/reprozip/reprozip/setup.py) egg_info for package reprozip
    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "/tmp/venv/src/reprozip/reprozip/setup.py", line 22, in <module>
        with open('README.rst') as fp:
    IOError: [Errno 2] No such file or directory: 'README.rst'
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 17, in <module>

  File "/tmp/venv/src/reprozip/reprozip/setup.py", line 22, in <module>

    with open('README.rst') as fp:

IOError: [Errno 2] No such file or directory: 'README.rst'

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/venv/src/reprozip
Storing debug log for failure in /home/remram/.pip/pip.log
(venv)remram@vebian:/tmp$

I'm not sure what more I could uninstall or upgrade.

[Ivoz]

Many setup.pys use something like this to anchor where they want to be reading files from. It's not really a requirement that a setup.py has to be called from its own directory.

[remram44]

The fact is that no one would ever run the setup.py from another directory. pip itself runs from the package's directory, it just forgets when subdirectory= is used.

What are the official setup.py requirements? Taking a second to look at matplotlib and IPython, I can see that they make the same assumption I did.

[remram44]

Hmm, this appears to have been fixed on develop with d3acfe08 (see #1600 #1601). That was from March 11. When is the next release?

[Ivoz]

https://github.com/pypa/pip/milestones/1.6

[jnpkrn]

It's not just about anchoring the referenced files, also some import trickery can be involved, etc. Would be nice if the change in develop branch made to a release.