search

conda-forge / conda-forge.github.io

The conda-forge website.
https://conda-forge.org
BSD 3-Clause "New" or "Revised" License
132 stars 279 forks source link

rpy2 installation linking problems #234

Closed jbwhit closed 7 years ago

jbwhit commented 8 years ago

I was having problems that I couldn't diagnose, so I deleted all of miniconda/.jupyter/etc, and reinstalled.

The error

import rpy2
%load_ext rpy2.ipython
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-3-a69f80d0128e> in <module>()
----> 1 get_ipython().magic('load_ext rpy2.ipython')

/Users/jonathan/miniconda3/envs/problem/lib/python3.4/site-packages/IPython/core/interactiveshell.py in magic(self, arg_s)
   2161         magic_name, _, magic_arg_s = arg_s.partition(' ')
   2162         magic_name = magic_name.lstrip(prefilter.ESC_MAGIC)
-> 2163         return self.run_line_magic(magic_name, magic_arg_s)
   2164
   2165     #-------------------------------------------------------------------------

/Users/jonathan/miniconda3/envs/problem/lib/python3.4/site-packages/IPython/core/interactiveshell.py in run_line_magic(self, magic_name, line)
   2082                 kwargs['local_ns'] = sys._getframe(stack_depth).f_locals
   2083             with self.builtin_trap:
-> 2084                 result = fn(*args,**kwargs)
   2085             return result
   2086

<decorator-gen-64> in load_ext(self, module_str)

/Users/jonathan/miniconda3/envs/problem/lib/python3.4/site-packages/IPython/core/magic.py in <lambda>(f, *a, **k)
    191     # but it's overkill for just that one bit of state.
    192     def magic_deco(arg):
--> 193         call = lambda f, *a, **k: f(*a, **k)
    194
    195         if callable(arg):

/Users/jonathan/miniconda3/envs/problem/lib/python3.4/site-packages/IPython/core/magics/extension.py in load_ext(self, module_str)
     64         if not module_str:
     65             raise UsageError('Missing module name.')
---> 66         res = self.shell.extension_manager.load_extension(module_str)
     67
     68         if res == 'already loaded':

/Users/jonathan/miniconda3/envs/problem/lib/python3.4/site-packages/IPython/core/extensions.py in load_extension(self, module_str)
     82             if module_str not in sys.modules:
     83                 with prepended_to_syspath(self.ipython_extension_dir):
---> 84                     __import__(module_str)
     85             mod = sys.modules[module_str]
     86             if self._call_load_ipython_extension(mod):

/Users/jonathan/miniconda3/envs/problem/lib/python3.4/site-packages/rpy2/ipython/__init__.py in <module>()
----> 1 from .rmagic import load_ipython_extension

/Users/jonathan/miniconda3/envs/problem/lib/python3.4/site-packages/rpy2/ipython/rmagic.py in <module>()
     50 # numpy and rpy2 imports
     51
---> 52 import rpy2.rinterface as ri
     53 import rpy2.robjects as ro
     54 import rpy2.robjects.packages as rpacks

/Users/jonathan/miniconda3/envs/problem/lib/python3.4/site-packages/rpy2/rinterface/__init__.py in <module>()
     90 del(os)
     91
---> 92 from rpy2.rinterface._rinterface import (baseenv,
     93                                          emptyenv,
     94                                          endr,

ImportError: dlopen(/Users/jonathan/miniconda3/envs/problem/lib/python3.4/site-packages/rpy2/rinterface/_rinterface.so, 2): Library not loaded: @rpath/libicuuc.54.dylib
  Referenced from: /Users/jonathan/miniconda3/envs/problem/lib/python3.4/site-packages/rpy2/rinterface/_rinterface.so
  Reason: image not found

How I installed everything

These were the steps that I ran:

conda update conda
conda config --add channels conda-forge
packages='ipywidgets
jupyter
jupyter_contrib_nbextensions
jupyter_nbextensions_configurator
matplotlib
mkl
mpld3
notebook
numpy
pandas
pip
pymc
pyparsing
scikit-learn
scipy
seaborn
statsmodels
'
conda create --name rpy3 -c r r r-irkernel r-recommended r-essentials rpy2 python=3 $packages -y
source activate rpy3
jupyter notebook

My setup

$ cat .condarc
channels:
  - conda-forge
  - defaults
anaconda_upload: true

And

$ conda info -a
Current conda install:

             platform : osx-64
        conda version : 4.1.11
    conda-env version : 2.5.2
  conda-build version : not installed
       python version : 3.5.2.final.0
     requests version : 2.10.0
     root environment : /Users/jonathan/miniconda3  (writable)
  default environment : /Users/jonathan/miniconda3
     envs directories : /Users/jonathan/miniconda3/envs
        package cache : /Users/jonathan/miniconda3/pkgs
         channel URLs : https://conda.anaconda.org/conda-forge/osx-64/
                        https://conda.anaconda.org/conda-forge/noarch/
                        https://repo.continuum.io/pkgs/free/osx-64/
                        https://repo.continuum.io/pkgs/free/noarch/
                        https://repo.continuum.io/pkgs/pro/osx-64/
                        https://repo.continuum.io/pkgs/pro/noarch/
          config file : /Users/jonathan/.condarc
         offline mode : False
    is foreign system : False

# conda environments:
#
rpy3                     /Users/jonathan/miniconda3/envs/rpy3
root                  *  /Users/jonathan/miniconda3

sys.version: 3.5.2 |Continuum Analytics, Inc.| (defau...
sys.prefix: /Users/jonathan/miniconda3
sys.executable: /Users/jonathan/miniconda3/bin/python
conda location: /Users/jonathan/miniconda3/lib/python3.5/site-packages/conda
conda-build: None
conda-env: /Users/jonathan/miniconda3/bin/conda-env
user site dirs: ~/.local/lib/python3.5

CIO_TEST: <not set>
CONDA_DEFAULT_ENV: <not set>
CONDA_ENVS_PATH: <not set>
DYLD_LIBRARY_PATH: <not set>
PATH: /Users/jonathan/miniconda3/bin:/Users/jonathan/miniconda3/bin:/Library/Frameworks/Python.framework/Versions/Current/bin:/Users/jonathan/Library/Haskell/bin:/Library/Frameworks/Python.framework/Versions/Current/bin:/Users/jonathan/github/whelper:/Users/jonathan/github/fitcmp:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Library/TeX/texbin:/Users/jonathan/Dropbox/scripts:/Users/jonathan/Dropbox/scripts/bash:/Users/jonathan/bin:/Users/jonathan/github/AstroTools
PYTHONHOME: <not set>
PYTHONPATH: /Users/jonathan/github/CaliCompari/Python:/Users/jwhitmore/github/linfit:/Library/Frameworks/Python.framework/Versions/Current/bin/python:/Library/Frameworks/Python.framework/Versions/Current/lib:/Library/Frameworks/Python.framework/Versions/Current/bin/python:/Users/jonathan/github/AstroTools:

WARNING: could not import _license.show_info
# try:
# $ conda install -n root _license
jbwhit commented 8 years ago

Well, it's not a solution to the conda-forge real problem posted, but adding conda-forge (going w/ the default instead of conda-forge channel) as suggested in https://github.com/conda-forge/conda-forge.github.io/issues/232 fixes the problem superficially.

jakirkham commented 8 years ago

So conda-forge doesn't provide rpy2. That has to be an issue with package from defaults then. @mingwandroid, could you please take a look at this?

andrea-soto commented 8 years ago

Has this issue been resolved? I'm getting the same error with the following versions: Python 3.5.2 :: Anaconda 4.2.0 (x86_64) jupyter notebook 4.2.3 rpy2 2.8.3

nick-youngblut commented 7 years ago

I'm getting the same issue. My setup:

# Sever: Ubuntu 14.04 x86_64. 
# Anaconda (for python3.5) install
wget https://repo.continuum.io/archive/Anaconda3-4.2.0-Linux-x86_64.sh
bash Anaconda3-4.2.0-Linux-x86_64.sh -b -p `pwd`"/anaconda3"
conda update conda

# Creating environment
conda create -y -n test_py2 python=python=2.7.12
conda install -y -n test_py2  -c conda-forge jupyter_contrib_nbextensions
conda install -c r rpy2

I get a similar error as @jbwhit when I try %load_ext rpy2.ipython. Here's the traceback:

ImportError                               Traceback (most recent call last)
<ipython-input-1-691c6d73b073> in <module>()
----> 1 get_ipython().magic(u'load_ext rpy2.ipython')

/ebio/abt3_projects/methanogen_host_evo/anaconda3/envs/py2_ley0.4/lib/python2.7/site-packages/IPython/core/interactiveshell.pyc in magic(self, arg_s)
   2156         magic_name, _, magic_arg_s = arg_s.partition(' ')
   2157         magic_name = magic_name.lstrip(prefilter.ESC_MAGIC)
-> 2158         return self.run_line_magic(magic_name, magic_arg_s)
   2159 
   2160     #-------------------------------------------------------------------------

/ebio/abt3_projects/methanogen_host_evo/anaconda3/envs/py2_ley0.4/lib/python2.7/site-packages/IPython/core/interactiveshell.pyc in run_line_magic(self, magic_name, line)
   2077                 kwargs['local_ns'] = sys._getframe(stack_depth).f_locals
   2078             with self.builtin_trap:
-> 2079                 result = fn(*args,**kwargs)
   2080             return result
   2081 

<decorator-gen-62> in load_ext(self, module_str)

/ebio/abt3_projects/methanogen_host_evo/anaconda3/envs/py2_ley0.4/lib/python2.7/site-packages/IPython/core/magic.pyc in <lambda>(f, *a, **k)
    186     # but it's overkill for just that one bit of state.
    187     def magic_deco(arg):
--> 188         call = lambda f, *a, **k: f(*a, **k)
    189 
    190         if callable(arg):

/ebio/abt3_projects/methanogen_host_evo/anaconda3/envs/py2_ley0.4/lib/python2.7/site-packages/IPython/core/magics/extension.pyc in load_ext(self, module_str)
     35         if not module_str:
     36             raise UsageError('Missing module name.')
---> 37         res = self.shell.extension_manager.load_extension(module_str)
     38 
     39         if res == 'already loaded':

/ebio/abt3_projects/methanogen_host_evo/anaconda3/envs/py2_ley0.4/lib/python2.7/site-packages/IPython/core/extensions.pyc in load_extension(self, module_str)
     81             if module_str not in sys.modules:
     82                 with prepended_to_syspath(self.ipython_extension_dir):
---> 83                     __import__(module_str)
     84             mod = sys.modules[module_str]
     85             if self._call_load_ipython_extension(mod):

/ebio/abt3_projects/methanogen_host_evo/anaconda3/envs/py2_ley0.4/lib/python2.7/site-packages/rpy2/ipython/__init__.py in <module>()
----> 1 from .rmagic import load_ipython_extension

/ebio/abt3_projects/methanogen_host_evo/anaconda3/envs/py2_ley0.4/lib/python2.7/site-packages/rpy2/ipython/rmagic.py in <module>()
     50 # numpy and rpy2 imports
     51 
---> 52 import rpy2.rinterface as ri
     53 import rpy2.robjects as ro
     54 import rpy2.robjects.packages as rpacks

/ebio/abt3_projects/methanogen_host_evo/anaconda3/envs/py2_ley0.4/lib/python2.7/site-packages/rpy2/rinterface/__init__.py in <module>()
     90 del(os)
     91 
---> 92 from rpy2.rinterface._rinterface import (baseenv,
     93                                          emptyenv,
     94                                          endr,

ImportError: libicuuc.so.54: cannot open shared object file: No such file or directory

Any ideas on what's going on? Thanks

ocefpaf commented 7 years ago

Any ideas on what's going on? Thanks

Like it is said above, conda-forge does not package r2py (yet) and there is no way to ensure that the defaults version of it will work with the rest of the conda-forge stack.

In this case what you are experiencing is the mismatch if the icu version. defaults uses 54.* while conda--forge uses 56.*.

jakirkham commented 7 years ago

FYI there is an effort underway to add R to conda-forge, but may take a bit of time to complete.

jbwhit commented 7 years ago

I'm glad to hear that there's an effort to add R to conda-forge.

In the meantime, this is such an interesting thread.

I think I understand the problem: using conda-forge installs the versions of each package it knows about, including icu on version 56.*. Then, when installing a package that conda-forge doesn't know about (rpy2 for example), it will download it, but that package might no longer work because it relies on the conda defaults channel, which points to 54.* in this case.

Now, I can understand that this is unsolvable in the sense conda-forge can't reason about dependencies for things they don't know about, but to mark the issue as closed seems incorrect. It may be unsolvable, but this issue is a real one. People who have added the conda-forge channel will have otherwise working installs broken (in a really hard to diagnose way).

I'm not sure what to do in the meantime. Perhaps put a warning somewhere to watch out, and that this kind of thing can happen?

mingwandroid commented 7 years ago

IMHO there should be a bug called "conda defaults and conda-forge are incompatible" and all these things should be marked as duplicates of that and then closed.

ocefpaf commented 7 years ago

Indeed. I'll do it soon.

nick-youngblut commented 7 years ago

So... I'm not sure how to get rpy2 working. Loading rpy2.ipython still throws the error: ImportError: libicuuc.so.54: cannot open shared object file: No such file or directory. My conda environment has the following r-related packages: 

r                         3.3.2                  r3.3.2_0    r
r-assertthat              0.1                    r3.3.2_4    r
r-base                    3.3.2                         1    conda-forge
r-bh                      1.62.0_1               r3.3.2_0    r
r-boot                    1.3_18                 r3.3.2_0    r
r-class                   7.3_14                 r3.3.2_0    r
r-cluster                 2.0.5                  r3.3.2_0    r
r-codetools               0.2_15                 r3.3.2_0    r
r-colorspace              1.3_1                  r3.3.2_0    r
r-dbi                     0.5_1                  r3.3.2_0    r
r-dichromat               2.0_0                  r3.3.2_2    r
r-digest                  0.6.10                 r3.3.2_0    r
r-dplyr                   0.5.0                  r3.3.2_0    r
r-foreign                 0.8_67                 r3.3.2_0    r
r-ggplot2                 2.2.0                  r3.3.2_0    r
r-gtable                  0.2.0                  r3.3.2_0    r
r-kernsmooth              2.23_15                r3.3.2_0    r
r-labeling                0.3                    r3.3.2_2    r
r-lattice                 0.20_34                r3.3.2_0    r
r-lazyeval                0.2.0                  r3.3.2_0    r
r-magrittr                1.5                    r3.3.2_2    r
r-mass                    7.3_45                 r3.3.2_0    r
r-matrix                  1.2_7.1                r3.3.2_0    r
r-mgcv                    1.8_16                 r3.3.2_0    r
r-munsell                 0.4.3                  r3.3.2_0    r
r-nlme                    3.1_128                r3.3.2_0    r
r-nnet                    7.3_12                 r3.3.2_0    r
r-plyr                    1.8.4                  r3.3.2_0    r
r-r6                      2.2.0                  r3.3.2_0    r
r-rcolorbrewer            1.1_2                  r3.3.2_3    r
r-rcpp                    0.12.8                 r3.3.2_0    r
r-recommended             3.3.2                  r3.3.2_0    r
r-reshape2                1.4.2                  r3.3.2_0    r
r-rpart                   4.1_10                 r3.3.2_0    r
r-scales                  0.4.1                  r3.3.2_0    r
r-spatial                 7.3_11                 r3.3.2_0    r
r-stringi                 1.1.2                  r3.3.2_0    r
r-stringr                 1.1.0                  r3.3.2_0    r
r-survival                2.40_1                 r3.3.2_0    r
r-tibble                  1.2                    r3.3.2_0    r
r-tidyr                   0.6.0                  r3.3.2_0    r
readline                  6.2                           0
requests                  2.12.4                   py35_0
rpy2                      2.8.5              py35r3.3.2_2    r

If the issue is that I'm using the conda-forge version of r-base, then how do I change to a different version of r-base? I can't seem to changer-base to another channel (e.g., r):

$ conda install --override-channels -c r r-base
Fetching package metadata .....
Solving package specifications: ..........

# All requested packages already installed.
# packages in environment at /ebio/abt3_projects/methanogen_formate/anaconda3/envs/py3_ley0.4:
#
r-base                    3.3.2                         1    conda-forge

I can't even search for different versions of r-base with anaconda search:

$ anaconda search r-base
Using Anaconda API: https://api.anaconda.org
Run 'anaconda show <USER/PACKAGE>' to get more details:
Packages:
     Name                      |  Version | Package Types   | Platforms
     ------------------------- |   ------ | --------------- | ---------------
[ERROR]
Traceback (most recent call last):
  File "/ebio/abt3_projects/methanogen_formate/anaconda3/envs/py3_ley0.4/bin/anaconda", line 6, in <module>
    sys.exit(binstar_client.scripts.cli.main())
  File "/ebio/abt3_projects/methanogen_formate/anaconda3/envs/py3_ley0.4/lib/python3.5/site-packages/binstar_client/scripts/cli.py", line 88, in main
    description=__doc__, version=version)
  File "/ebio/abt3_projects/methanogen_formate/anaconda3/envs/py3_ley0.4/lib/python3.5/site-packages/binstar_client/scripts/cli.py", line 66, in binstar_main
    return args.main(args)
  File "/ebio/abt3_projects/methanogen_formate/anaconda3/envs/py3_ley0.4/lib/python3.5/site-packages/binstar_client/commands/search.py", line 15, in search
    pprint_packages(packages, access=False)
  File "/ebio/abt3_projects/methanogen_formate/anaconda3/envs/py3_ley0.4/lib/python3.5/site-packages/binstar_client/utils/pprint.py", line 70, in pprint_packages
    pprint_package(package, access, full_name, revision=revisions)
  File "/ebio/abt3_projects/methanogen_formate/anaconda3/envs/py3_ley0.4/lib/python3.5/site-packages/binstar_client/utils/pprint.py", line 34, in pprint_package
    package['conda_platforms'] = ', '.join(package['conda_platforms'])
TypeError: sequence item 2: expected str instance, NoneType found

Right now, I'm kind of stuck. Any help would be greatly appreciated!

ocefpaf commented 7 years ago

Right now, I'm kind of stuck. Any help would be greatly appreciated!

@nick-youngblut we cannot help you with a software that is not in conda-forge. Please read https://github.com/conda-forge/conda-forge.github.io/issues/234#issuecomment-27574955

You need to rely on defaults only for that.

mingwandroid commented 7 years ago

You need to remove conda-forge from your channels list in ~/.condarc then do conda update all, or just conda install -c r r-base to install what you need.

You should use conda search -c r r-base for the search you are attempting (though this does look like you've run into a bug in anaconda-client).

nick-youngblut commented 7 years ago

Thanks for the help! Removing conda-forge from my channels in my ~/.condarc and then running conda install -c r r-base worked.

I'm finding that conda can be tricky to use: there's always the pitfall of installing the 'wrong' version of a package, and then it can be hard to change the package version (e.g., having to manually remove a channel from ~/.condarc). Still, for most bioinformatics software, conda (& especially bioconda) makes installation extremely simple.

malcook commented 7 years ago

Despite this issue being closed, I thought I would chime in since things are changed since @nick-youngblut experience given that the bioconda team seems to have gotten updated R into the conda-forge. Given that, it is preferable NOT to use the 'r' channel at all, and rather to have conda-forge come before default. With that setup, so far, for me, many bioconda installs are, as nick wrote "extremely simple".

vsbuffalo commented 7 years ago

I'm having this same issue, and I agree with the following from above:

but to mark the issue as closed seems incorrect. It may be unsolvable, but this issue is a real one. People who have added the conda-forge channel will have otherwise working installs broken (in a really hard to diagnose way).

As far as I can tell, if a user needs rpy2, and anything in conda-forge, this issue occurs. rpy2 is an essential part of the Python data stack for many. This problem means that I can't use conda at all for my current project, which seems to be a serious issue.

I'm happy to provide more information if needed on my specific case. But I think ignoring this issue outright is biting a lot of folks using conda for science.

mingwandroid commented 7 years ago

Thanks for the offer of information, but I'm fairly sure it's just same old vanilla binary incompatibility.

If (as I do) you would like to be able to use Rpy2 in conda-forge you can contribute a recipe, hopefully with my patches so that it works on Windows correctly. I would do it myself but I'm a bit busy:

https://github.com/conda/conda-recipes/tree/master/python/rpy2

Cheers!

ocefpaf commented 7 years ago

It has been a while that conda-forge has rpy2. I'd like to know if that works for you there.

mingwandroid commented 7 years ago

Ah well there you go!

Thanks @ocefpaf. I wonder if the problem is now that defaults has R 3.4.1 and conda-forge doesn't (or does it?)

ocefpaf commented 7 years ago

Thanks @ocefpaf. I wonder if the problem is now that defaults has R 3.4.1 and conda-forge doesn't (or does it?)

We do https://github.com/conda-forge/r-base-feedstock/blob/master/recipe/meta.yaml#L3

We may need to re-render rpy2 though. I'll check that soon.

mingwandroid commented 7 years ago

Ah, in that case @bgruening of @johanneskoester, if you have time, you may want to merge across https://github.com/conda/conda-recipes/tree/master/r-packages/r-rjava (and the rest?!).

vsbuffalo commented 7 years ago

Hm, so I'm trying to figure out precisely what's going wrong. I also need r-dbplyr which isn't on conda-forge, and I'm thinking that may change something conda-forge's rpy2 needs. Here is the full environment I'm trying to create (for completeness and other folks Googling similar things):

$ cat ~/.condarc
channels:
  - bioconda
  - conda-forge
  - defaults
$  conda create -n fwpdy11 python=3.6 r-base=3.4.1 rpy2 r-tidyverse r-dbplyr r-devtools gsl=1.16 fwdpy11 pylibseq ipython jupyter numpy pandas tqdm msprime scipy

Then I try to get this working from iPython:

%load_ext rpy2.ipython
import fwdpy11 as fp11
import numpy as np
import pandas as pd
from tqdm import tqdm_notebook
import msprime as msp
import libsequence as ls

Often, even if I get an environment somehow configured that works with the above, there's issues with dynamic libraries that crop up with (again from iPython):

%%R 
library(tidyverse)
library(dbplyr)

Conda Info:

conda info
Current conda install:

               platform : linux-64
          conda version : 4.3.23
       conda is private : False
      conda-env version : 4.3.23
    conda-build version : not installed
         python version : 3.6.1.final.0
       requests version : 2.14.2
       root environment : /home/vinceb/anaconda3  (writable)
    default environment : /home/vinceb/anaconda3/envs/fwpdy11
       envs directories : /home/vinceb/anaconda3/envs
                          /home/vinceb/.conda/envs
          package cache : /home/vinceb/anaconda3/pkgs
                          /home/vinceb/.conda/pkgs
           channel URLs : https://conda.anaconda.org/bioconda/linux-64
                          https://conda.anaconda.org/bioconda/noarch
                          https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.continuum.io/pkgs/free/linux-64
                          https://repo.continuum.io/pkgs/free/noarch
                          https://repo.continuum.io/pkgs/r/linux-64
                          https://repo.continuum.io/pkgs/r/noarch
                          https://repo.continuum.io/pkgs/pro/linux-64
                          https://repo.continuum.io/pkgs/pro/noarch
            config file : /home/vinceb/.condarc
             netrc file : None
           offline mode : False
             user-agent : conda/4.3.23 requests/2.14.2 CPython/3.6.1 Linux/3.10.0-514.26.2.el7.x86_64 CentOS Linux/7.3.1611 glibc/2.17
                UID:GID : 1002:1002
mingwandroid commented 7 years ago

@vsbuffalo, the most useful information here for us to help diagnose this is:

conda list --show-channel-urls

isuruf commented 7 years ago

rpy2 is not available on conda-forge for r=3.4.1. It's available only for r=3.3.2

ocefpaf commented 7 years ago

@vsbuffalo you have 3 channels there and I have a slide (#8) for you :wink:

I am re-rendering now to get rpy2 for r=3.4.1 in the channel. But we would still need a PR for r-dbplyr.

vsbuffalo commented 7 years ago

@ocefpaf heh, nice slides. If we get r-dbplyr into conda-forge, that might solve it. It's a dependency to r-tidyverse whenever database backends are used. I'll take a look at how this is done, maybe it's something I could help with.

@mingwandroid, sure here that is:

# packages in environment at /home/vinceb/anaconda3/envs/fwpdy11:
#
blas                      1.1                    openblas    conda-forge
bleach                    1.4.2                    py36_0    bioconda
bzip2                     1.0.6                         1    conda-forge
ca-certificates           2017.7.27.1                   0    conda-forge
cairo                     1.14.6                        4    conda-forge
certifi                   2017.7.27.1              py36_0    conda-forge
curl                      7.54.1                        0    conda-forge
dbus                      1.10.22                       0    conda-forge
decorator                 4.1.2                    py36_0    conda-forge
entrypoints               0.2.3                    py36_1    conda-forge
expat                     2.2.1                         0    conda-forge
fontconfig                2.12.1                        4    conda-forge
freetype                  2.7                           1    conda-forge
fwdpy11                   0.1.3a1          py36_gsl1.16_0    bioconda
gettext                   0.19.7                        1    conda-forge
glib                      2.51.4                        0    conda-forge
gmp                       6.1.2                         0    conda-forge
graphite2                 1.3.9                         0    conda-forge
gsl                       1.16                          0    conda-forge
gst-plugins-base          1.8.0                         0    conda-forge
gstreamer                 1.8.0                         2    conda-forge
h5py                      2.7.0               np113py36_1    conda-forge
harfbuzz                  1.4.3                         0    conda-forge
hdf5                      1.8.18                        1    conda-forge
html5lib                  0.999999999              py36_0    conda-forge
htslib                    1.5                           0    bioconda
icu                       58.1                          1    conda-forge
ipykernel                 4.6.1                    py36_0    conda-forge
ipython                   6.1.0                    py36_0    conda-forge
ipython_genutils          0.2.0                    py36_0    conda-forge
ipywidgets                7.0.0                    py36_0    conda-forge
jedi                      0.10.2                   py36_0    conda-forge
jinja2                    2.9.6                    py36_0    conda-forge
jpeg                      9b                            0    conda-forge
jsonschema                2.6.0                    py36_0    conda-forge
jupyter                   1.0.0                    py36_0    conda-forge
jupyter_client            5.1.0                    py36_0    conda-forge
jupyter_console           5.1.0                    py36_0    conda-forge
jupyter_core              4.3.0                    py36_0    conda-forge
krb5                      1.14.2                        0    conda-forge
libffi                    3.2.1                         3    conda-forge
libgcc                    5.2.0                         0    defaults
libgfortran               3.0.0                         1    defaults
libiconv                  1.14                          4    conda-forge
libpng                    1.6.28                        1    conda-forge
libsequence               1.9.1               htslib1.5_1    bioconda
libsodium                 1.0.10                        0    conda-forge
libssh2                   1.8.0                         1    conda-forge
libtiff                   4.0.6                         7    conda-forge
libxcb                    1.12                          1    conda-forge
libxml2                   2.9.4                         4    conda-forge
markupsafe                1.0                      py36_0    conda-forge
mistune                   0.7.4                    py36_0    conda-forge
msprime                   0.4.0            py36_gsl1.16_2    bioconda
nbconvert                 5.2.1                    py36_1    conda-forge
nbformat                  4.3.0                    py36_0    conda-forge
ncurses                   5.9                          10    conda-forge
notebook                  5.0.0                    py36_0    conda-forge
numpy                     1.13.1          py36_blas_openblas_200  [blas_openblas]  conda-forge
openblas                  0.2.19                        2    conda-forge
openssl                   1.0.2l                        0    conda-forge
pandas                    0.20.3                   py36_1    conda-forge
pandoc                    1.19.2                        0    conda-forge
pandocfilters             1.4.1                    py36_0    conda-forge
pango                     1.40.4                        0    conda-forge
pcre                      8.39                          0    conda-forge
pexpect                   4.2.1                    py36_0    conda-forge
pickleshare               0.7.4                    py36_0    conda-forge
pip                       9.0.1                    py36_0    conda-forge
pixman                    0.34.0                        0    conda-forge
prompt_toolkit            1.0.15                   py36_0    conda-forge
ptyprocess                0.5.2                    py36_0    conda-forge
pygments                  2.2.0                    py36_0    conda-forge
pylibseq                  0.1.9.post0              py36_0    bioconda
pyparsing                 2.2.0                    py36_0    conda-forge
pyqt                      5.6.0                    py36_4    conda-forge
python                    3.6.2                         0    conda-forge
python-dateutil           2.6.1                    py36_0    conda-forge
pytz                      2017.2                   py36_0    conda-forge
pyzmq                     16.0.2                   py36_2    conda-forge
qt                        5.6.2                         3    conda-forge
qtconsole                 4.3.1                    py36_0    conda-forge
r-assertthat              0.1                    r3.4.1_0    conda-forge
r-base                    3.4.1                         0    conda-forge
r-bh                      1.62.0_1               r3.4.1_0    conda-forge
r-bindr                   0.1                    r3.4.1_0    defaults
r-bindrcpp                0.2                    r3.4.1_0    defaults
r-broom                   0.4.2                  r3.4.1_0    defaults
r-cellranger              1.1.0                  r3.4.1_0    conda-forge
r-colorspace              1.3_2                  r3.4.1_0    conda-forge
r-curl                    2.4                    r3.4.1_0    conda-forge
r-dbi                     0.6_1                  r3.4.1_0    conda-forge
r-dbplyr                  1.1.0                  r3.4.1_0    defaults
r-devtools                1.13.2                 r3.4.1_0    conda-forge
r-dichromat               2.0_0                  r3.4.1_0    conda-forge
r-digest                  0.6.10                 r3.4.1_0    conda-forge
r-dplyr                   0.7.0                  r3.4.1_0    defaults
r-forcats                 0.2.0                  r3.4.1_0    conda-forge
r-foreign                 0.8_67                 r3.4.1_0    conda-forge
r-ggplot2                 2.2.0                  r3.4.1_0    conda-forge
r-git2r                   0.19.0                 r3.4.1_0    conda-forge
r-glue                    1.1.1                  r3.4.1_0    defaults
r-gtable                  0.2.0                  r3.4.1_0    conda-forge
r-haven                   1.0.0                  r3.4.1_0    conda-forge
r-hms                     0.3                    r3.4.1_0    conda-forge
r-httr                    1.2.1                  r3.4.1_0    conda-forge
r-jsonlite                1.4                    r3.4.1_0    conda-forge
r-labeling                0.3                    r3.4.1_0    conda-forge
r-lattice                 0.20_34                r3.4.1_0    conda-forge
r-lazyeval                0.2.0                  r3.4.1_0    conda-forge
r-lubridate               1.6.0                  r3.4.1_0    conda-forge
r-magrittr                1.5                    r3.4.1_0    conda-forge
r-mass                    7.3_45                 r3.4.1_0    conda-forge
r-memoise                 1.1.0                  r3.4.1_0    conda-forge
r-mime                    0.5                    r3.4.1_0    conda-forge
r-mnormt                  1.5_5                  r3.4.1_0    conda-forge
r-modelr                  0.1.0                  r3.4.1_0    conda-forge
r-munsell                 0.4.3                  r3.4.1_0    conda-forge
r-nlme                    3.1_131                r3.4.1_0    conda-forge
r-openssl                 0.9.6                  r3.4.1_1    conda-forge
r-pkgconfig               2.0.1                  r3.4.1_0    defaults
r-plogr                   0.1_1                  r3.4.1_0    conda-forge
r-plyr                    1.8.4                  r3.4.1_0    conda-forge
r-psych                   1.7.3.21               r3.4.1_0    conda-forge
r-purrr                   0.2.2                         1    bioconda
r-r6                      2.2.0                  r3.4.1_0    conda-forge
r-rcolorbrewer            1.1_2                  r3.4.1_0    conda-forge
r-rcpp                    0.12.10                r3.4.1_0    conda-forge
r-readr                   1.1.0                  r3.4.1_0    conda-forge
r-readxl                  1.0.0                  r3.4.1_0    conda-forge
r-rematch                 1.0.1                  r3.4.1_0    conda-forge
r-reshape2                1.4.2                  r3.4.1_0    conda-forge
r-rlang                   0.1.2                  r3.4.1_0    conda-forge
r-rstudioapi              0.6                    r3.4.1_0    conda-forge
r-rvest                   0.3.2                  r3.4.1_0    conda-forge
r-scales                  0.4.1                  r3.4.1_0    conda-forge
r-selectr                 0.3_1                  r3.4.1_0    conda-forge
r-stringi                 1.1.2                  r3.4.1_0    conda-forge
r-stringr                 1.2.0                  r3.4.1_0    conda-forge
r-tibble                  1.3.3                  r3.4.1_0    defaults
r-tidyr                   0.6.1                  r3.4.1_0    conda-forge
r-tidyverse               1.1.1                  r3.4.1_0    conda-forge
r-whisker                 0.3_2                  r3.4.1_0    conda-forge
r-withr                   1.0.2                  r3.4.1_0    conda-forge
r-xml2                    1.1.1                  r3.4.1_0    conda-forge
readline                  6.2                           0    conda-forge
rpy2                      2.8.6              py36r3.4.1_2    defaults
scipy                     0.19.1          py36_blas_openblas_202  [blas_openblas]  conda-forge
setuptools                36.2.2                   py36_0    conda-forge
simplegeneric             0.8.1                    py36_0    conda-forge
singledispatch            3.4.0.3                  py36_0    conda-forge
sip                       4.18                     py36_1    conda-forge
six                       1.10.0                   py36_1    conda-forge
sqlite                    3.13.0                        1    conda-forge
svgwrite                  1.1.6                    py36_0    bioconda
tbb                       4.4_20150728                  0    bioconda
terminado                 0.6                      py36_0    conda-forge
testpath                  0.3.1                    py36_0    conda-forge
tk                        8.5.19                        2    conda-forge
tornado                   4.5.1                    py36_0    conda-forge
tqdm                      4.7.2                    py36_0    bioconda
traitlets                 4.3.2                    py36_0    conda-forge
wcwidth                   0.1.7                    py36_0    conda-forge
webencodings              0.5                      py36_0    conda-forge
wheel                     0.29.0                   py36_0    conda-forge
widgetsnbextension        3.0.2                    py36_0    conda-forge
xorg-libxau               1.0.8                         3    conda-forge
xorg-libxdmcp             1.1.2                         3    conda-forge
xz                        5.2.3                         0    conda-forge
zeromq                    4.2.1                         1    conda-forge
zlib                      1.2.8                         3    conda-forge
mingwandroid commented 7 years ago

@vsbuffalo, please, if you have time do help! Here is the recipe used to build the package on defaults:

https://github.com/conda/conda-recipes/tree/master/r-packages/r-dbplyr

bgruening commented 7 years ago

I added a PR for r-dbplyr with https://github.com/conda-forge/staged-recipes/pull/3861.

vsbuffalo commented 7 years ago

So for completeness, here is an update. First, I can rework my code so r-dbplyr is no longer necessary (but then new issues crop up). Then, the following works:

$ conda create -n default-fwdpy11-2 python=3.6 r-base=3.4.1 r-tidyverse r-devtools r-rsqlite gsl=1.16 fwdpy11 pylibseq ipython jupyter numpy pandas tqdm msprime scipy 
$ conda install -c conda-forge -n default-fwdpy11 rpy2

while trying to install it all in one go, e.g.:

$ conda create -n default-fwdpy11-2 python=3.6 r-base=3.4.1 r-tidyverse r-devtools r-rsqlite gsl=1.16 fwdpy11 pylibseq ipython jupyter numpy pandas tqdm msprime scipy rpy2

does not work.

Essentially, I can get a reproducible-ish environment through iteratively crafting with the command line interface. In general, issues are avoided and conda is quite useful if one minimizes the channels used (I think this is what @ocefpaf's slides suggest) as this creates dependency nightmares. As a conda-beginner, it seems avoiding default and r entirely would be useful, but not always feasible — this is where I got bitten. AFAIK, there's still not a way to install r-dplyr=0.7 from r or default without re-causing the original issue. I can try writing a recipe, as this is a rather important release (the behavior of collect() changes drastically).

mingwandroid commented 7 years ago

As someone who works on r and defaults (pedantically r is a part of defaults now) as well as on conda-forge, I would like to know what is missing from r \ defaults that prevents you being able to use them.

I want all the options to be viable basically (and mixable in future).

ocefpaf commented 7 years ago

it seems avoiding default and r entirely would be useful, but not always feasible

The long-term goal is to make conda-forge and defaults as compatible as possible to avoid such headaches. And you are correct, we need at least those two, but if we can avoid other channels we should.

r-base 3.4.1 was added recently to conda-forge, right when we had the token issue, and we need to re-render the recipes to get the packages. rpy2, for example, is stuck b/c of that (missing some dependencies). See https://github.com/conda-forge/rpy2-feedstock/pull/2

I'll see what I can next weekend, but I cannot promise anything (king of buried in work here).

If you want to help take a look at https://conda-forge.org/docs/conda_smithy.html and try to send re-rendering PRs to the feedtocks we need to be available for r 3.4.1, like r-ggplot. We do have a bot to do that but it goes really slow b/c it can only process a few feedstocks per day to avoid bringing our CIs down.

isuruf commented 7 years ago

I want all the options to be viable basically (and mixable in future).

For mixing conda-forge and defaults, one issue is that defaults is using a broken gettext version when compiling r-base. gettext=0.19.8 accidentally bumped the soversion and this was reverted in gettext=0.19.8.1. conda-forge is using 0.19.8.1 while defaults is using 0.19.8

mingwandroid commented 7 years ago

@isuruf, thanks.

bgruening commented 7 years ago

r-dbplyr is now available.