SebastianHollizeck commented 4 years ago

Hi,

now that you changed to a setup.py, i ran into some issues when trying to install. most packages need a specific version to work with your tool, otherwise it fails, but the setup process actually uses the newest available instead.

i had to manually install a few of the deps to make it work.

scikit-learn==0.20 zipp==1.2 sphinx==1.8.3 ipython==5.1 joblib==0.10

and it required cython and to call the program with the python -m hatchet notation, i also had to install 'configparser'

It would be good if you could add that as specific requirements, otherwise its quite a pain to actually install.

Cheers, Sebastian

simozacca commented 4 years ago

Thank you so much for this useful feedback on the new automatic installation!

@vineetbansal Could you please look into this?

vineetbansal commented 4 years ago

@SebastianHollizeck .. thanks for the feedback. This is great info. Let's take these one by one:

scikit-learn: We do want to use the latest version of scikit-learn which is compatible with Python 2. This is version 0.20.4, which is pulled in automatically by pip. So I'd like to avoid pinning down the version of scikit-learn unless you know for a fact that you're having issues with the latest version that pip has chosen for you.
zipp/sphinx/ipython/joblib - None of these packages are directly needed by hatchet, so you won't find them in setup.py. It's entirely possible that other packages recursively depend on them, in which case they would have pinned down the dependencies themselves. If they're failing to do so and a pip install . is failing, then yes, I agree that we should pin them while the package authors try to fix things on their end.
cython is already in setup.py. This is actually needed as a build dependency by bnpy which fails to mention this in their package structure. This should cease to be a problem in a couple of weeks. If the latest cython version is crashing for you then do let us know.
Great point about configparser - that should have been mentioned in setup.py. We'll add that shortly (and keep this open until we fix it).

In general, I don't want to pin down any dependencies unless we know there's a problem with the latest versions applicable for Python 2.7. If other packages are more restrictive in their requirements than that's perfectly fine. So I'm curious to know about your experience with just doing a pip install . or pip install -e . on a brand-new python 2.7 environment. If things are installing fine but then failing at runtime that would belong in this thread too. The hope is to not have to do anything beyond just that one command. If you are, then let us know your setup (o/s, conda version, pip version etc) and we'll try to include that as part of our CI testing.

SebastianHollizeck commented 4 years ago

I am happy to help, i know how much effort it is to bundle a package.

I created a new venv and then tried to install, but had those issues that i talked about

i can just show what the issues were for me for each of the package I mentioned

zipp

(hatchet) [shollizeck@vmpr-res-cluster1 hatchet]$ python -m hatchet BBot -h
Traceback (most recent call last):
  File "/usr/lib64/python2.7/runpy.py", line 151, in _run_module_as_main
    mod_name, loader, code, fname = _get_module_details(mod_name)
  File "/usr/lib64/python2.7/runpy.py", line 109, in _get_module_details
    return _get_module_details(pkg_main_name)
  File "/usr/lib64/python2.7/runpy.py", line 101, in _get_module_details
    loader = get_loader(mod_name)
  File "/usr/lib64/python2.7/pkgutil.py", line 464, in get_loader
    return find_loader(fullname)
  File "/usr/lib64/python2.7/pkgutil.py", line 474, in find_loader
    for importer in iter_importers(fullname):
  File "/usr/lib64/python2.7/pkgutil.py", line 430, in iter_importers
    __import__(pkg)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/hatchet-0.1.0-py2.7-linux-x86_64.egg/hatchet/__init__.py", line 4, in <module>
    from importlib_resources import path
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/importlib_resources-3.0.0-py2.7.egg/importlib_resources/__init__.py", line 5, in <module>
    from ._common import (
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/importlib_resources-3.0.0-py2.7.egg/importlib_resources/_common.py", line 9, in <module>
    from ._compat import (
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/importlib_resources-3.0.0-py2.7.egg/importlib_resources/_compat.py", line 42, in <module>
    from zipp import Path as ZipPath  £ type: ignore
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/zipp-3.2.0-py2.7.egg/zipp.py", line 226
    def open(self, mode='r', *args, pwd=None, **kwargs):
                                      ^
SyntaxError: invalid syntax

sphinx

Installed /home/shollizeck/hatchet/lib/python2.7/site-packages/hatchet-0.1.0-py2.7-linux-x86_64.egg
Processing dependencies for hatchet==0.1.0
Searching for sphinx>=1.8.3
Reading https://pypi.org/simple/sphinx/
Downloading https://files.pythonhosted.org/packages/57/58/48268b16bf3e6e8288c4c6f3d500e4dd1ca0210289a5be8366bd6d2e6088/Sphinx-3.2.1.tar.gz£sha256=321d6d9b16fa381a5306e5a0b76cd48ffbc588e6340059a729c6fdd66087e0e8
Best match: Sphinx 3.2.1
Processing Sphinx-3.2.1.tar.gz
Writing /tmp/easy_install-ZqQNyU/Sphinx-3.2.1/setup.cfg
Running Sphinx-3.2.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-ZqQNyU/Sphinx-3.2.1/egg-dist-tmp-I2rHaA
Traceback (most recent call last):
  File "setup.py", line 97, in <module>
    'dev': ['pytest', 'mock']
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/__init__.py", line 162, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib64/python2.7/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/usr/lib64/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/install.py", line 67, in run
    self.do_egg_install()
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/install.py", line 117, in do_egg_install
    cmd.run(show_deprecation=False)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 424, in run
    self.easy_install(spec, not self.no_deps)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 666, in easy_install
    return self.install_item(None, spec, tmpdir, deps, True)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 713, in install_item
    self.process_distribution(spec, dist, deps)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 758, in process_distribution
    [requirement], self.local_index, self.easy_install
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/pkg_resources/__init__.py", line 782, in resolve
    replace_conflicting=replace_conflicting
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1065, in best_match
    return self.obtain(req, installer)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1077, in obtain
    return installer(requirement)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 685, in easy_install
    return self.install_item(spec, dist.location, tmpdir, deps)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 711, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 896, in install_eggs
    return self.build_and_install(setup_script, setup_base)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1164, in build_and_install
    self.run_setup(setup_script, setup_base, args)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1150, in run_setup
    run_setup(setup_script, args)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/sandbox.py", line 253, in run_setup
    raise
  File "/usr/lib64/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
    yield
  File "/usr/lib64/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/sandbox.py", line 166, in save_modules
    saved_exc.resume()
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/sandbox.py", line 141, in resume
    six.reraise(type, exc, self._tb)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/sandbox.py", line 154, in save_modules
    yield saved
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
    yield
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/sandbox.py", line 250, in run_setup
    _execfile(setup_script, ns)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/sandbox.py", line 45, in _execfile
    exec(code, globals, locals)
  File "/tmp/easy_install-ZqQNyU/Sphinx-3.2.1/setup.py", line 8, in <module>
    from setuptools import setup, Extension
  File "/tmp/easy_install-ZqQNyU/Sphinx-3.2.1/sphinx/__init__.py", line 20, in <module>
  File "/tmp/easy_install-ZqQNyU/Sphinx-3.2.1/sphinx/deprecation.py", line 32
    def deprecated_alias(modname: str, objects: Dict[str, object],
                                ^
SyntaxError: invalid syntax

scikit-learn

Processing dependencies for hatchet==0.1.0
Searching for scikit-learn
Reading https://pypi.org/simple/scikit-learn/
Downloading https://files.pythonhosted.org/packages/aa/f6/75297be19f48b7a8c2577753a3a700f98fc4db49d0e5ed3820dd8dee43d4/scikit-learn-0.23.2.tar.gz£sha256=20766f515e6cd6f954554387dfae705d93c7b544ec0e6c6a5d8e006f6f7ef480
Best match: scikit-learn 0.23.2
Processing scikit-learn-0.23.2.tar.gz
Writing /tmp/easy_install-g8Ndii/scikit-learn-0.23.2/setup.cfg
Running scikit-learn-0.23.2/setup.py -q bdist_egg --dist-dir /tmp/easy_install-g8Ndii/scikit-learn-0.23.2/egg-dist-tmp-jGO4lx
Partial import of sklearn during the build process.
Traceback (most recent call last):
  File "setup.py", line 97, in <module>
    'dev': ['pytest', 'mock']
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/__init__.py", line 162, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib64/python2.7/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/usr/lib64/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/install.py", line 67, in run
    self.do_egg_install()
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/install.py", line 117, in do_egg_install
    cmd.run(show_deprecation=False)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 424, in run
    self.easy_install(spec, not self.no_deps)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 666, in easy_install
    return self.install_item(None, spec, tmpdir, deps, True)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 713, in install_item
    self.process_distribution(spec, dist, deps)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 758, in process_distribution
    [requirement], self.local_index, self.easy_install
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/pkg_resources/__init__.py", line 782, in resolve
    replace_conflicting=replace_conflicting
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1065, in best_match
    return self.obtain(req, installer)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1077, in obtain
    return installer(requirement)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 685, in easy_install
    return self.install_item(spec, dist.location, tmpdir, deps)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 711, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 896, in install_eggs
    return self.build_and_install(setup_script, setup_base)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1164, in build_and_install
    self.run_setup(setup_script, setup_base, args)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1150, in run_setup
    run_setup(setup_script, args)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/sandbox.py", line 253, in run_setup
    raise
  File "/usr/lib64/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
    yield
  File "/usr/lib64/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/sandbox.py", line 166, in save_modules
    saved_exc.resume()
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/sandbox.py", line 141, in resume
    six.reraise(type, exc, self._tb)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/sandbox.py", line 154, in save_modules
    yield saved
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
    yield
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/sandbox.py", line 250, in run_setup
    _execfile(setup_script, ns)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/sandbox.py", line 45, in _execfile
    exec(code, globals, locals)
  File "/tmp/easy_install-g8Ndii/scikit-learn-0.23.2/setup.py", line 304, in <module>

  File "/tmp/easy_install-g8Ndii/scikit-learn-0.23.2/setup.py", line 290, in setup_package

RuntimeError: Scikit-learn requires Python 3.6 or later. The current Python version is 2.7.5 installed in /home/shollizeck/hatchet/bin/python.

cython

Searching for bnpy
Reading https://pypi.org/simple/bnpy/
Downloading https://files.pythonhosted.org/packages/16/61/234331f36c1781aa04a6307ca54f73a1eb9d140bb11656195c4484b14cb4/bnpy-0.1.6.tar.gz£sha256=6d6e4c2ca46c6b0cb331f1365933895728cf7333cef95d58249c7c01667b54d0
Best match: bnpy 0.1.6
Processing bnpy-0.1.6.tar.gz
Writing /tmp/easy_install-A1dSQP/bnpy-0.1.6/setup.cfg
Running bnpy-0.1.6/setup.py -q bdist_egg --dist-dir /tmp/easy_install-A1dSQP/bnpy-0.1.6/egg-dist-tmp-ERp1iK
Traceback (most recent call last):
  File "setup.py", line 97, in <module>
    'dev': ['pytest', 'mock']
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/__init__.py", line 162, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib64/python2.7/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/usr/lib64/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/install.py", line 67, in run
    self.do_egg_install()
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/install.py", line 117, in do_egg_install
    cmd.run(show_deprecation=False)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 424, in run
    self.easy_install(spec, not self.no_deps)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 666, in easy_install
    return self.install_item(None, spec, tmpdir, deps, True)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 713, in install_item
    self.process_distribution(spec, dist, deps)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 758, in process_distribution
    [requirement], self.local_index, self.easy_install
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/pkg_resources/__init__.py", line 782, in resolve
    replace_conflicting=replace_conflicting
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1065, in best_match
    return self.obtain(req, installer)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1077, in obtain
    return installer(requirement)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 685, in easy_install
    return self.install_item(spec, dist.location, tmpdir, deps)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 711, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 896, in install_eggs
    return self.build_and_install(setup_script, setup_base)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1164, in build_and_install
    self.run_setup(setup_script, setup_base, args)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1150, in run_setup
    run_setup(setup_script, args)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/sandbox.py", line 253, in run_setup
    raise
  File "/usr/lib64/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
    yield
  File "/usr/lib64/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/sandbox.py", line 166, in save_modules
    saved_exc.resume()
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/sandbox.py", line 141, in resume
    six.reraise(type, exc, self._tb)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/sandbox.py", line 154, in save_modules
    yield saved
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
    yield
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/sandbox.py", line 250, in run_setup
    _execfile(setup_script, ns)
  File "/home/shollizeck/hatchet/lib/python2.7/site-packages/setuptools/sandbox.py", line 45, in _execfile
    exec(code, globals, locals)
  File "/tmp/easy_install-A1dSQP/bnpy-0.1.6/setup.py", line 3, in <module>
    import sys
ImportError: No module named Cython.Distutils

freeze

this is how i got it to install in the end

(hatchet) [shollizeck@vmpr-res-cluster1 ~]$ pip freeze
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/£python-2-support pip 21.0 will remove support for this functionality.
alabaster==0.7.12
Babel==2.8.0
backports.functools-lru-cache==1.6.1
backports.shutil-get-terminal-size==1.0.0
biopython==1.76
bnpy==0.1.6
certifi==2020.6.20
chardet==3.0.4
configparser==4.0.2
contextlib2==0.6.0.post1
cycler==0.10.0
Cython==0.29.21
decorator==4.4.2
dill==0.3.2
docutils==0.16
enum34==1.1.10
futures==3.3.0
hatchet==0.1.0
idna==2.10
imagesize==1.2.0
importlib-resources==3.0.0
ipython==5.1.0
ipython-genutils==0.2.0
Jinja2==2.11.2
joblib==0.10.0
kiwisolver==1.1.0
MarkupSafe==1.1.1
matplotlib==2.2.5
matplotlib-venn==0.11.5
memory-profiler==0.57.0
multiprocess==0.70.10
munkres==1.0.12
numexpr==2.7.1
numpy==1.16.6
opencv-python==4.2.0.32
packaging==20.4
pandas==0.24.2
pathlib2==2.3.5
pexpect==4.8.0
pickleshare==0.7.5
prompt-toolkit==1.0.18
psutil==5.7.2
ptyprocess==0.6.0
Pygments==2.5.2
pyparsing==2.4.7
pysam==0.16.0.1
python-dateutil==2.8.1
pytz==2020.1
requests==2.24.0
scandir==1.10.0
scikit-learn==0.20.0
scipy==1.2.3
seaborn==0.9.1
simplegeneric==0.8.1
singledispatch==3.4.0.3
six==1.15.0
snowballstemmer==2.0.0
Sphinx==1.8.3
sphinx-gallery==0.8.1
sphinxcontrib-websupport==1.1.2
subprocess.run==0.0.8
subprocess32==3.5.4
traitlets==4.3.3
typing==3.7.4.3
urllib3==1.25.10
wcwidth==0.2.5
zipp==1.2.0

vineetbansal commented 4 years ago

Thanks for the details @SebastianHollizeck.

It seems to me that all the above issues (except possibly cython) are resulting from pip trying to install Python 3 packages in a Python 2 environment, .i.e. it's not respecting the python_requires directive in setup.py. So you should see the same issue with zipp, for example, when trying to install importlib_resources in a brand new Python 2 environment, taking hatchet totally out of the picture.

I can only speculate that you may inadvertently picking up a very old version of pip (or maybe setuptools?) that is unaware of the python_requires directive (in which case a pip install -U pip should fix it, but don't do so just yet since I'm interested in reproducing your error).

Can I ask you to provide (after having activated the hatchet venv like you're doing in your case):

The path and version of python (which python and python --version)
The path and version of pip (which pip and pip --version)
The version of setuptools (python -c "import setuptools; print(setuptools.__version__)")

Thanks for your patience!

SebastianHollizeck commented 4 years ago

Sure thing, this is the output in virtual environment

(hatchet) [shollizeck@vmpr-res-cluster1 ~]$ which python
~/hatchet/bin/python
(hatchet) [shollizeck@vmpr-res-cluster1 ~]$ python --version
Python 2.7.5
(hatchet) [shollizeck@vmpr-res-cluster1 ~]$ which pip
~/hatchet/bin/pip
(hatchet) [shollizeck@vmpr-res-cluster1 ~]$ pip --version
pip 9.0.1 from /home/shollizeck/hatchet/lib/python2.7/site-packages (python 2.7)
(hatchet) [shollizeck@vmpr-res-cluster1 ~]$ python -c "import setuptools; print(setuptools.__version__)"
28.8.0

vineetbansal commented 4 years ago

Thanks @SebastianHollizeck .. I was able to reproduce your install errors using the versions you specified. The issue was twofold:

your pip is recent enough, but not setuptools. Any version below 40.8.0 wouldn't pick up our pyproject.toml, the preferred way to set up build environments for a few years now. I've set up some checks now so that pip install will refuse to proceed unless you have that min. version of setuptools. We cannot set up a reliable test for pip itself though, which will work both in virtual environments as well as conda environments, so we've also updated our install instructions to execute pip install -U pip and pip install -U setuptools right after activating your virtual environment to avoid issues like these.
our instructions asked you to execute python setup.py install. This happens to be less than ideal for Python 2 (since historical python 2 versions of upstream packages including bnpy etc. are set up with the assumption that pip will be used). We're thus now recommending that you do a pip install . (or pip install -e . for develop mode), and have updated our instructions.

So if you don't mind trashing your virtual env, recreating and activating it, and trying the steps again:

python2 -m virtualenv env
source env/bin/activate
pip install -U pip
pip install -U setuptools
pip install . (or pip install -e . if you expect to make changes to the source and observe it's effects). You may have to precede this with CXXFLAGS=-pthreads (also in the instructions) in case the build is failing on your platform.

A lot of these issues should go away once we have the Python 3 version of the code (hopefully soon).

simozacca commented 4 years ago

@SebastianHollizeck could you please confirm that the issue has been solved?

SebastianHollizeck commented 4 years ago

Hi,

i tried what you said to check if that fixes it, but the cython problem persists.

[shollizeck@vmpr-res-cluster1 ~]$ python2 -m virtualenv new
New python executable in /home/shollizeck/new/bin/python2
Also creating executable in /home/shollizeck/new/bin/python
Installing setuptools, pip, wheel...done.
[shollizeck@vmpr-res-cluster1 ~]$ source new/bin/activate
(new) [shollizeck@vmpr-res-cluster1 ~]$ pip install -U pip
Cache entry deserialization failed, entry ignored
Collecting pip
  Using cached https://files.pythonhosted.org/packages/4e/5f/528232275f6509b1fff703c9280e58951a81abe24640905de621c9f81839/pip-20.2.3-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 9.0.1
    Uninstalling pip-9.0.1:
      Successfully uninstalled pip-9.0.1
Successfully installed pip-20.2.3
(new) [shollizeck@vmpr-res-cluster1 ~]$ pip install -U setuptools
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Collecting setuptools
  Downloading setuptools-44.1.1-py2.py3-none-any.whl (583 kB)
     |████████████████████████████████| 583 kB 55 kB/s 
Installing collected packages: setuptools
  Attempting uninstall: setuptools
    Found existing installation: setuptools 28.8.0
    Uninstalling setuptools-28.8.0:
      Successfully uninstalled setuptools-28.8.0
Successfully installed setuptools-44.1.1
(new) [shollizeck@vmpr-res-cluster1 ~]$pip install .
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Processing /home/shollizeck/hatchet_deps/hatchet
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting matplotlib-venn
  Downloading matplotlib-venn-0.11.5.zip (40 kB)
     |████████████████████████████████| 40 kB 285 kB/s 
Collecting cython
  Using cached Cython-0.29.21-cp27-cp27mu-manylinux1_x86_64.whl (1.9 MB)
Collecting matplotlib
  Downloading matplotlib-2.2.5-cp27-cp27mu-manylinux1_x86_64.whl (12.8 MB)
     |████████████████████████████████| 12.8 MB 2.6 kB/s 
Collecting pandas
  Downloading pandas-0.24.2-cp27-cp27mu-manylinux1_x86_64.whl (10.1 MB)
     |████████████████████████████████| 10.1 MB 21 kB/s 
Collecting pysam
  Downloading pysam-0.16.0.1-cp27-cp27mu-manylinux1_x86_64.whl (9.2 MB)
     |████████████████████████████████| 9.2 MB 15 kB/s 
Collecting bnpy
  Downloading bnpy-0.1.6.tar.gz (1.8 MB)
     |████████████████████████████████| 1.8 MB 31 kB/s 
    ERROR: Command errored out with exit status 1:
     command: /home/shollizeck/new/bin/python2 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-Ynw67Q/bnpy/setup.py'"'"'; __file__='"'"'/tmp/pip-install-Ynw67Q/bnpy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-RMYUbc
         cwd: /tmp/pip-install-Ynw67Q/bnpy/
    Complete output (5 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-Ynw67Q/bnpy/setup.py", line 3, in <module>
        from Cython.Distutils import build_ext
    ImportError: No module named Cython.Distutils
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Like i said, I have a working installation, so it doesnt bother me too much, but it would be good for the next person using it ;)

vineetbansal commented 4 years ago

Thanks @SebastianHollizeck .. great to know the rest of the dependency issues were fixed. The cython issue is kind of tricky to properly fix, since bnpy fails to list it as one of its build dependencies. Till that issue is resolved on their end (hopefully soon), we'll modify our instructions to ask the user to do a pip install cython before trying to install the package.

simozacca commented 4 years ago

Thanks @SebastianHollizeck .. great to know the rest of the dependency issues were fixed. The cython issue is kind of tricky to properly fix, since bnpy fails to list it as one of its build dependencies. Till that issue is resolved on their end (hopefully soon), we'll modify our instructions to ask the user to do a pip install cython before trying to install the package.

@vineetbansal Is this a requirement to add in step 5.?

vineetbansal commented 4 years ago

@simozacca .. yes, added this step in readme immediately after step 5. Merged in PR #19. Closing this here.

raphael-group / hatchet

installation requires specific versions (which are no specified) #15

zipp

sphinx

scikit-learn

cython

freeze