Conflicting Python resources and virtualenvs

[jwdebelius]

I tried to run Daniel's notebooks tonight (11/10). The ipymd release I got (ipymd==0.1.1) required IPython 4.0 or greater. The code in mod2_pcoa.py would not run with IPython > 4.0:

Traceback (most recent call last):
  File "/Users/jwdebelius/.virtualenvs/test_env/bin/mod2_pcoa.py", line 4, in <module>
    __import__('pkg_resources').require('americangut==0.0.1')
  File "/Users/jwdebelius/.virtualenvs/test_env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3018, in <module>
    working_set = WorkingSet._build_master()
  File "/Users/jwdebelius/.virtualenvs/test_env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 614, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/Users/jwdebelius/.virtualenvs/test_env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 627, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/Users/jwdebelius/.virtualenvs/test_env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 805, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: IPython<4.0.0

I did downgrade my matplotlib to 1.4, and this is a function which calls seaborn, but the error isn't related to those packages.

[wasade]

I thought setup.py specified IPython<4.0.0? On Nov 10, 2015 11:36 PM, "J W Debelius" notifications@github.com wrote:

I tried to run Daniel's notebooks tonight (11/10). The ipymd release I got (ipymd==0.1.1) required IPython 4.0 or greater. The code in mod2_pcoa.py would not run with IPython > 4.0:

Traceback (most recent call last): File "/Users/jwdebelius/.virtualenvs/test_env/bin/mod2_pcoa.py", line 4, in import('pkg_resources').require('americangut==0.0.1') File "/Users/jwdebelius/.virtualenvs/test_env/lib/python2.7/site-packages/pkg_resources/init.py", line 3018, in working_set = WorkingSet._build_master() File "/Users/jwdebelius/.virtualenvs/test_env/lib/python2.7/site-packages/pkg_resources/init.py", line 614, in _build_master return cls._build_from_requirements(requires) File "/Users/jwdebelius/.virtualenvs/test_env/lib/python2.7/site-packages/pkg_resources/init.py", line 627, in _build_from_requirements dists = ws.resolve(reqs, Environment()) File "/Users/jwdebelius/.virtualenvs/test_env/lib/python2.7/site-packages/pkg_resources/init.py", line 805, in resolve raise DistributionNotFound(req) pkg_resources.DistributionNotFound: IPython<4.0.0

I did downgrade my matplotlib to 1.4, and this is a function which calls seaborn, but the error isn't related to those packages.

— Reply to this email directly or view it on GitHub https://github.com/biocore/American-Gut/issues/180.

[jwdebelius]

To run ipymd as installed by setup.py I had to get the latest version of Jupyter.

On Nov 11, 2015, at 6:49 AM, Daniel McDonald notifications@github.com wrote:

I thought setup.py specified IPython<4.0.0? On Nov 10, 2015 11:36 PM, "J W Debelius" notifications@github.com wrote:

I tried to run Daniel's notebooks tonight (11/10). The ipymd release I got (ipymd==0.1.1) required IPython 4.0 or greater. The code in mod2_pcoa.py would not run with IPython > 4.0:

Traceback (most recent call last): File "/Users/jwdebelius/.virtualenvs/test_env/bin/mod2_pcoa.py", line 4, in import('pkg_resources').require('americangut==0.0.1') File "/Users/jwdebelius/.virtualenvs/test_env/lib/python2.7/site-packages/pkg_resources/init.py", line 3018, in working_set = WorkingSet._build_master() File "/Users/jwdebelius/.virtualenvs/test_env/lib/python2.7/site-packages/pkg_resources/init.py", line 614, in _build_master return cls._build_from_requirements(requires) File "/Users/jwdebelius/.virtualenvs/test_env/lib/python2.7/site-packages/pkg_resources/init.py", line 627, in _build_from_requirements dists = ws.resolve(reqs, Environment()) File "/Users/jwdebelius/.virtualenvs/test_env/lib/python2.7/site-packages/pkg_resources/init.py", line 805, in resolve raise DistributionNotFound(req) pkg_resources.DistributionNotFound: IPython<4.0.0

I did downgrade my matplotlib to 1.4, and this is a function which calls seaborn, but the error isn't related to those packages.

— Reply to this email directly or view it on GitHub https://github.com/biocore/American-Gut/issues/180.

— Reply to this email directly or view it on GitHub.

[wasade]

Can you try emulating how packages are installed in travis? Possible setup.py is out of sync To run ipymd as installed by setup.py I had to get the latest version of Jupyter.

On Nov 11, 2015, at 6:49 AM, Daniel McDonald notifications@github.com wrote:

I thought setup.py specified IPython<4.0.0? On Nov 10, 2015 11:36 PM, "J W Debelius" notifications@github.com wrote:

I tried to run Daniel's notebooks tonight (11/10). The ipymd release I got (ipymd==0.1.1) required IPython 4.0 or greater. The code in mod2_pcoa.py would not run with IPython > 4.0:

Traceback (most recent call last): File "/Users/jwdebelius/.virtualenvs/test_env/bin/mod2_pcoa.py", line 4, in import('pkg_resources').require('americangut==0.0.1') File "/Users/jwdebelius/.virtualenvs/test_env/lib/python2.7/site-packages/pkg_resources/init.py", line 3018, in working_set = WorkingSet._build_master() File "/Users/jwdebelius/.virtualenvs/test_env/lib/python2.7/site-packages/pkg_resources/init.py", line 614, in _build_master return cls._build_from_requirements(requires) File "/Users/jwdebelius/.virtualenvs/test_env/lib/python2.7/site-packages/pkg_resources/init.py", line 627, in _build_from_requirements dists = ws.resolve(reqs, Environment()) File "/Users/jwdebelius/.virtualenvs/test_env/lib/python2.7/site-packages/pkg_resources/init.py", line 805, in resolve raise DistributionNotFound(req) pkg_resources.DistributionNotFound: IPython<4.0.0

I did downgrade my matplotlib to 1.4, and this is a function which calls seaborn, but the error isn't related to those packages.

— Reply to this email directly or view it on GitHub https://github.com/biocore/American-Gut/issues/180.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/biocore/American-Gut/issues/180#issuecomment-155829347.

[jwdebelius]

I installed it according to travis, although I use virtualenv instead of conda as my enviroment manager. I installed the additional packages (numpy, lxml, and h5py), and then re-installed in the same directory.

I continue to get the error.

[wasade]

Sounds like your environment is not sane as the conda_requirements.txt clearly lists ipython<4.0.0. Suggest deleting your environment and recreating.

[jwdebelius]

I've built and rebuilt the environment several times using the directions provided in virtualenv. No change.

I think its critical we continue to support both conda and virtualenv with this package set up.

[wasade]

If the directions or install requirements are wrong, then they need to be fixed. However, it is not clear what is wrong about them given this thread which makes investigation impossible. This does deploy on many systems including multiple distributions of Linux and OS X.

There is nothing here that indicates virtualenv is unsupported, or that an environment manager is even required. There is nothing about this repo that is conda specific. I strongly recommend switching to conda regardless though as it vastly simplifies environment management.

Can you verify that there is nothing modifying $PYTHONPATH in your environment prior to creating or activating an environment in any of your login scripts (e.g., ~/.bash_profile)? And, can you verify that executables such as ipython are not in your $PATH prior to creating or activating an environment?

On Wed, Nov 11, 2015 at 3:59 PM, J W Debelius notifications@github.com wrote:

I've built and rebuilt the environment several times using the directions provided in virtualenv. No change.

I think its critical we continue to support both conda and virtualenv with this package set up.

— Reply to this email directly or view it on GitHub https://github.com/biocore/American-Gut/issues/180#issuecomment-155948792 .

[wasade]

I cannot recreate this issue. I used an empty environment, installed numpy and h5py separately as is necessary for qiime via pypi, and did "pip install -e ." I suspect that there is confusion in your environment external to a manager.

On Wed, Nov 11, 2015 at 4:26 PM, Daniel T. McDonald < Daniel.Mcdonald@colorado.edu> wrote:

If the directions or install requirements are wrong, then they need to be fixed. However, it is not clear what is wrong about them given this thread which makes investigation impossible. This does deploy on many systems including multiple distributions of Linux and OS X.

There is nothing here that indicates virtualenv is unsupported, or that an environment manager is even required. There is nothing about this repo that is conda specific. I strongly recommend switching to conda regardless though as it vastly simplifies environment management.

Can you verify that there is nothing modifying $PYTHONPATH in your environment prior to creating or activating an environment in any of your login scripts (e.g., ~/.bash_profile)? And, can you verify that executables such as ipython are not in your $PATH prior to creating or activating an environment?

On Wed, Nov 11, 2015 at 3:59 PM, J W Debelius notifications@github.com wrote:

I've built and rebuilt the environment several times using the directions provided in virtualenv. No change.

I think its critical we continue to support both conda and virtualenv with this package set up.

— Reply to this email directly or view it on GitHub https://github.com/biocore/American-Gut/issues/180#issuecomment-155948792 .