search

jobovy / gaia_tools

Tools for working with the @ESAGaia data and related data sets
MIT License
60 stars 19 forks source link

suggesting install changes on ubuntu #5

Closed lgbouma closed 7 years ago

lgbouma commented 7 years ago

Hi!

On ubuntu systems (maybe different on other operating systems?), when installing via sudo python setup.py install from a virtualenv (like conda, which probably most people are doing) none of the variables, aliases, etc carry over, and running the setup script as root fails.

The work-around for this was (from the appropriate venv):

$(venv_name) export GAIA_TOOLS_DATA=~/.GAIA_TOOLS_DATA # or whatever
$(venv_name) python setup.py install --user

which worked fine.

Might be worth mentioning this in the README? (root access shouldn't be necessary in the first place...)

Thanks!

jobovy commented 7 years ago

Thanks for the feedback! Could you post how the setup script fails? I don't quite understand what you mean by 'none of the variables, aliases, etc carry over', because no variables are necessary to install the package... root access also shouldn't be necessary (I don't install this as root myself).

lgbouma commented 7 years ago

I mean it in the sense of (from the terminal):

> (my_venv_name) luke@computer_name$ which python
/home/luke/miniconda3/envs/my_venv_name/bin/python
> (my_venv_name) luke@computer_name$ sudo -i
> root@computer_name:$ which python
/usr/bin/python

In other words sudo python foobar.py runs whatever python is installed in the root environment. If your root environment's python doesn't have the needed dependencies (numpy, astropy, fitsio, etc), while your venv python does (and you wanted to be using the venv python in the first place, but you ran sudo python setup.py install blindly following the README, like I did at first), the installation will fail.

The specific first error I got was passed via healpy first, but this isn't really the point. (Associated log pasted, in case you're interested)

(sci27) luke@crispy:~/Dropbox/proj/gaia_tools$ sudo python setup.py install
[sudo] password for luke: 
running install
/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py:2512: PEP440Warning: 'matlabengineforpython (R2015b)' is being parsed as a legacy, non PEP 440, version. You may find odd behavior and sort order. In particular it will be sorted as less than 0.0. It is recommend to migrate to PEP 440 compatible versions.
  PEP440Warning,
/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py:2512: PEP440Warning: 'python-apt (0.9.3.11build1)' is being parsed as a legacy, non PEP 440, version. You may find odd behavior and sort order. In particular it will be sorted as less than 0.0. It is recommend to migrate to PEP 440 compatible versions.
  PEP440Warning,
running bdist_egg
running egg_info
writing requirements to gaia_tools.egg-info/requires.txt
writing gaia_tools.egg-info/PKG-INFO
writing top-level names to gaia_tools.egg-info/top_level.txt
writing dependency_links to gaia_tools.egg-info/dependency_links.txt
reading manifest file 'gaia_tools.egg-info/SOURCES.txt'
writing manifest file 'gaia_tools.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/gaia_tools
copying gaia_tools/__init__.py -> build/lib.linux-x86_64-2.7/gaia_tools
creating build/lib.linux-x86_64-2.7/gaia_tools/load
copying gaia_tools/load/path.py -> build/lib.linux-x86_64-2.7/gaia_tools/load
copying gaia_tools/load/download.py -> build/lib.linux-x86_64-2.7/gaia_tools/load
copying gaia_tools/load/__init__.py -> build/lib.linux-x86_64-2.7/gaia_tools/load
creating build/lib.linux-x86_64-2.7/gaia_tools/xmatch
copying gaia_tools/xmatch/__init__.py -> build/lib.linux-x86_64-2.7/gaia_tools/xmatch
creating build/lib.linux-x86_64-2.7/gaia_tools/select
copying gaia_tools/select/tgasSelect.py -> build/lib.linux-x86_64-2.7/gaia_tools/select
copying gaia_tools/select/__init__.py -> build/lib.linux-x86_64-2.7/gaia_tools/select
copying gaia_tools/select/2massc_hp5.txt -> build/lib.linux-x86_64-2.7/gaia_tools/select
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/gaia_tools
creating build/bdist.linux-x86_64/egg/gaia_tools/select
copying build/lib.linux-x86_64-2.7/gaia_tools/select/tgasSelect.py -> build/bdist.linux-x86_64/egg/gaia_tools/select
copying build/lib.linux-x86_64-2.7/gaia_tools/select/2massc_hp5.txt -> build/bdist.linux-x86_64/egg/gaia_tools/select
copying build/lib.linux-x86_64-2.7/gaia_tools/select/__init__.py -> build/bdist.linux-x86_64/egg/gaia_tools/select
creating build/bdist.linux-x86_64/egg/gaia_tools/load
copying build/lib.linux-x86_64-2.7/gaia_tools/load/path.py -> build/bdist.linux-x86_64/egg/gaia_tools/load
copying build/lib.linux-x86_64-2.7/gaia_tools/load/download.py -> build/bdist.linux-x86_64/egg/gaia_tools/load
copying build/lib.linux-x86_64-2.7/gaia_tools/load/__init__.py -> build/bdist.linux-x86_64/egg/gaia_tools/load
copying build/lib.linux-x86_64-2.7/gaia_tools/__init__.py -> build/bdist.linux-x86_64/egg/gaia_tools
creating build/bdist.linux-x86_64/egg/gaia_tools/xmatch
copying build/lib.linux-x86_64-2.7/gaia_tools/xmatch/__init__.py -> build/bdist.linux-x86_64/egg/gaia_tools/xmatch
byte-compiling build/bdist.linux-x86_64/egg/gaia_tools/select/tgasSelect.py to tgasSelect.pyc
byte-compiling build/bdist.linux-x86_64/egg/gaia_tools/select/__init__.py to __init__.pyc
byte-compiling build/bdist.linux-x86_64/egg/gaia_tools/load/path.py to path.pyc
byte-compiling build/bdist.linux-x86_64/egg/gaia_tools/load/download.py to download.pyc
byte-compiling build/bdist.linux-x86_64/egg/gaia_tools/load/__init__.py to __init__.pyc
byte-compiling build/bdist.linux-x86_64/egg/gaia_tools/__init__.py to __init__.pyc
byte-compiling build/bdist.linux-x86_64/egg/gaia_tools/xmatch/__init__.py to __init__.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying gaia_tools.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying gaia_tools.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying gaia_tools.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying gaia_tools.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying gaia_tools.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
gaia_tools.select.tgasSelect: module references __file__
creating 'dist/gaia_tools-0.1-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing gaia_tools-0.1-py2.7.egg
removing '/usr/local/lib/python2.7/dist-packages/gaia_tools-0.1-py2.7.egg' (and everything under it)
creating /usr/local/lib/python2.7/dist-packages/gaia_tools-0.1-py2.7.egg
Extracting gaia_tools-0.1-py2.7.egg to /usr/local/lib/python2.7/dist-packages
gaia-tools 0.1 is already the active version in easy-install.pth

Installed /usr/local/lib/python2.7/dist-packages/gaia_tools-0.1-py2.7.egg
Processing dependencies for gaia-tools==0.1
Searching for healpy
Reading https://pypi.python.org/simple/healpy/
/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py:2512: PEP440Warning: 'healpy (github-devel)' is being parsed as a legacy, non PEP 440, version. You may find odd behavior and sort order. In particular it will be sorted as less than 0.0. It is recommend to migrate to PEP 440 compatible versions.
  PEP440Warning,
/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py:2512: PEP440Warning: 'healpy-github (devel)' is being parsed as a legacy, non PEP 440, version. You may find odd behavior and sort order. In particular it will be sorted as less than 0.0. It is recommend to migrate to PEP 440 compatible versions.
  PEP440Warning,
Best match: healpy 1.10.3
Downloading https://pypi.python.org/packages/b0/e6/7e5002721095fad75680fe14bc6f8593e8ea853b345dd5dfbb263eb03ed5/healpy-1.10.3.tar.gz#md5=6491777d1bcbd36d356551bf240d3a2f
Processing healpy-1.10.3.tar.gz
Writing /tmp/easy_install-IxL70r/healpy-1.10.3/setup.cfg
Running healpy-1.10.3/setup.py -q bdist_egg --dist-dir /tmp/easy_install-IxL70r/healpy-1.10.3/egg-dist-tmp-aODUPW
/usr/lib/python2.7/distutils/extension.py:133: UserWarning: Unknown Extension options: 'cython_directives'
  warnings.warn(msg)
Traceback (most recent call last):
  File "setup.py", line 18, in <module>
    install_requires=['numpy','astropy','fitsio','tqdm','healpy']
  File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/usr/lib/python2.7/dist-packages/setuptools/command/install.py", line 67, in run
    self.do_egg_install()
  File "/usr/lib/python2.7/dist-packages/setuptools/command/install.py", line 117, in do_egg_install
    cmd.run()
  File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 397, in run
    self.easy_install(spec, not self.no_deps)
  File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 627, in easy_install
    return self.install_item(None, spec, tmpdir, deps, True)
  File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 678, in install_item
    self.process_distribution(spec, dist, deps)
  File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 726, in process_distribution
    [requirement], self.local_index, self.easy_install
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 799, in resolve
    dist = best[req.key] = env.best_match(req, ws, installer)
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1049, in best_match
    return self.obtain(req, installer)
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1061, in obtain
    return installer(requirement)
  File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 646, in easy_install
    return self.install_item(spec, dist.location, tmpdir, deps)
  File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 676, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 861, in install_eggs
    return self.build_and_install(setup_script, setup_base)
  File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 1089, in build_and_install
    self.run_setup(setup_script, setup_base, args)
  File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 1075, in run_setup
    run_setup(setup_script, args)
  File "/usr/lib/python2.7/dist-packages/setuptools/sandbox.py", line 240, in run_setup
    raise
  File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/lib/python2.7/dist-packages/setuptools/sandbox.py", line 193, in setup_context
    yield
  File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/lib/python2.7/dist-packages/setuptools/sandbox.py", line 164, in save_modules
    saved_exc.resume()
  File "/usr/lib/python2.7/dist-packages/setuptools/sandbox.py", line 139, in resume
    compat.reraise(type, exc, self._tb)
  File "/usr/lib/python2.7/dist-packages/setuptools/sandbox.py", line 152, in save_modules
    yield saved
  File "/usr/lib/python2.7/dist-packages/setuptools/sandbox.py", line 193, in setup_context
    yield
  File "/usr/lib/python2.7/dist-packages/setuptools/sandbox.py", line 237, in run_setup
    DirectorySandbox(setup_dir).run(runner)
  File "/usr/lib/python2.7/dist-packages/setuptools/sandbox.py", line 267, in run
    return func()
  File "/usr/lib/python2.7/dist-packages/setuptools/sandbox.py", line 236, in runner
    _execfile(setup_script, ns)
  File "/usr/lib/python2.7/dist-packages/setuptools/sandbox.py", line 46, in _execfile
    exec(code, globals, locals)
  File "/tmp/easy_install-IxL70r/healpy-1.10.3/setup.py", line 458, in <module>

  File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/usr/lib/python2.7/dist-packages/setuptools/command/bdist_egg.py", line 152, in run
    self.run_command("egg_info")
  File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/usr/lib/python2.7/dist-packages/setuptools/command/egg_info.py", line 179, in run
    self.find_sources()
  File "/usr/lib/python2.7/dist-packages/setuptools/command/egg_info.py", line 206, in find_sources
    mm.run()
  File "/usr/lib/python2.7/dist-packages/setuptools/command/egg_info.py", line 290, in run
    self.add_defaults()
  File "/usr/lib/python2.7/dist-packages/setuptools/command/egg_info.py", line 319, in add_defaults
    sdist.add_defaults(self)
  File "/usr/lib/python2.7/dist-packages/setuptools/command/sdist.py", line 130, in add_defaults
    build_ext = self.get_finalized_command('build_ext')
  File "/usr/lib/python2.7/distutils/cmd.py", line 312, in get_finalized_command
    cmd_obj.ensure_finalized()
  File "/usr/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized
    self.finalize_options()
  File "/tmp/easy_install-IxL70r/healpy-1.10.3/setup.py", line 353, in finalize_options

AttributeError: 'dict' object has no attribute '__NUMPY_SETUP__'
jobovy commented 7 years ago

Thanks for the follow-up. It might be that using sudo -E works? Or first running python setup.py build and then sudo python setup.py install? Anyway, I don't consider this problem to be specific to gaia_tools, because it must be an issue for any sudo python setup.py install on this kind of setup.

lgbouma commented 7 years ago

Tried both and they each give the same sort of error as that posted above. (Although the man page description of sudo -E makes it seem like it should work).

I agree that it's not a gaia_tools problem specifically -- I just thought it was worth suggesting given that the python setup.py install --user flag is a workaround to having to use sudo python setup.py install in the first place.

jobovy commented 7 years ago

Okay, I've added the python setup.py install --user (even though the method with --prefix= also does not require sudo).