rpy2.rinterface on Windows gives DLL import error

[ijstokes]

As reported on Anaconda ML, after doing conda install -c r rpy2 on Windows with Anaconda3, rpy2.rinterface cannot be imported due to a DLL import error:

In [2]: from rpy2 import rinterface
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
<ipython-input-2-40203d26f554> in <module>()
----> 1 from rpy2 import rinterface

C:\Users\ijsto\Anaconda3\lib\site-packages\rpy2\rinterface\__init__.py in <module>()
     85 
     86 if sys.platform == 'win32':
---> 87     _load_r_dll(R_HOME)
     88 
     89 # cleanup the namespace

C:\Users\ijsto\Anaconda3\lib\site-packages\rpy2\rinterface\__init__.py in _load_r_dll(r_home)
     62     if r_bin not in os.environ.get('PATH'):
     63         os.environ['PATH'] = ';'.join((os.environ.get('PATH'), r_bin, r_mod))
---> 64     ctypes.CDLL(r_dll)
     65 
     66 

C:\Users\ijsto\Anaconda3\lib\ctypes\__init__.py in __init__(self, name, mode, handle, use_errno, use_last_error)
    345 
    346         if handle is None:
--> 347             self._handle = _dlopen(self._name, mode)
    348         else:
    349             self._handle = handle

OSError: [WinError 126] The specified module could not be found

This may be related to #605 or #759.

I've tried a few other things but to no avail (just ended up with different problems).

So right now I think rpy2 (the essential bridge between Python and R in Anaconda) is broken.

[mingwandroid]

@Hi @ijstokes, I have not been able to reproduce this. I tried on Python 3.4 and Python 3.5 in both my existing miniconda installations and in a brand new installation of Anaconda3 4.2, both from the console and from ipython (though not every possible combination was tested).

The recipe for rpy2 was initially based on your one from here: https://github.com/ContinuumIO/anaconda-issues/issues/759#issuecomment-221467795 and I run all of the import tests when I build rpy2:

https://github.com/mingwandroid/conda-recipes/blob/master/python/rpy2/meta.yaml#L58-L70

Please show the output of conda list.

[mingwandroid]

Looking into this more, if I run from the Jupyter Notebook Start Menu shortcut then indeed the problem is reproducible (I tend to live entirely on the command line, even on Windows).

The problem happens because environments are not activated in the python script that menuinst uses to execute conda-installed sotware:

https://github.com/ContinuumIO/menuinst/blob/master/cwp.py

.. instead we just augment the path (incorrectly, it's missing the Library/mingw-w64/bin folder). Changing that part of cwp.py to the following:

env = os.environ.copy()
env['PATH'] = os.path.pathsep.join([
        prefix,
        join(prefix, "Scripts"),
        join(prefix, "Library", "bin"),
        join(prefix, "Library", "mingw-w64", "bin"),
        env['PATH'],
])

.. avoids this bug. However, I want to support macOS and Linux with menuinst someday and if I do that, then for rpy2 to work (or indeed any R notebook) when launched in this way, we absolutely need to activate environments correctly.

@ilanschnell, your thoughts on why we don't activate them in cwp.py would be appreciated.

[ijstokes]

@mingwandroid those errors were, I believe, from the command line. I started cmd.exe and then typed ipython, that's it (NOTE: it wasn't the Anaconda Command Prompt that I started from). But when I try it now (CLI or from IPython QtConsole in Navigator) the command works! I have no recollection of what could have changed in the past 48 hours -- very little I'd say! Attached is my conda list -e output.

win10_ana42.txt

In case it is helpful, I also had these windows pop up:

[mingwandroid]

Hi @ijstokes, did you see my most recent comment? The google groups posting does imply launching from Jupyter Notebook and I guess most Windows users will use the Start Menu shortcuts, and when doing this I could reproduce it. Can you verify that is the case and that my modification to cwp.py fixes it for you also?

[ijstokes]

@mingwandroid See other issues I've just submitted this morning. Anaconda doesn't uninstall or install nicely #1131 . I'm working through those issues now, as a precursor to reproducing the problem and attempting your suggested fix.

[ijstokes]

OK, my start menu is fracked and I don't know how to fix it. So that is an obstacle to reproducing the problem from the Start Menu.

However, here is what I've tried:

1. Uninstall everything using the standard Windows uninstaller (as mentioned in #1131 I don't believe this does everything it needs to).
2. Go into my home directory and delete a bunch of directories (as per feature request in #1126)
3. Reboot
4. Install Anaconda 4.2 (64-bit, Python 3.5) for Windows into Win10 system in my home directory ("Just for me"). Tick the two check boxes: "Add to PATH" and "Make default Python".
5. conda install -c r rpy2 to install rpy2 and its dependencies into my root environment.
6. Start up a Python interpreter by various means (cmd.exe then python, then ipython; QtConsole; Jupyter Notebook; IPython start menu entry; Anaconda Command Prompt).
7. Execute import rpy2 -- it always worked without an error.
8. Execute from rpy2 import rinterface -- it always failed with variations on the DLL load stack trace.

[mingwandroid]

Install Anaconda 4.2 (64-bit, Python 3.5) for Windows into Win10 system in my home directory ("Just for me"). Tick the two check boxes: "Add to PATH" and "Make default Python".

This doesn't, from a conda perspective, activate an environment. You get three PATH entries added ($CONDA_PREFIX, $CONDA_PREFIX/Scripts and $CONDA_PREFIX/Library/bin), whereas activating an environment involves adding also $CONDA_PREFIX/Library/mingw-w64/bin. It also runs the activate scripts for any packages that have them (which can set other necessary package-specific environment variables, or do whatever else is necessary).

We could change it so that $CONDA_PREFIX/Library/mingw-w64/bin is also added to PATH by the installer, but due to DLL hell and env. var length limits, personally I would rather remove the Add to PATH option altogether.

If you want to use a conda-compatible cmd.exe, then currently the only ways to do that are to use the Anaconda Prompt or else to run activate.bat from cmd.exe.

[ijstokes]

@csoja could this protocol (listed above) be something that gets added to the manual test sequence managed by @MulticastMatt testing team? It is unclear to me if the problem is in rpy2 or Anaconda itself. Some of @mingwandroid comments suggests this problem is baked in to Anaconda.

[ijstokes]

@mingwandroid there is no cwp.py file in my C:\Users\ijsto\Anaconda3\Lib\site-packages\menuinst directory, so I don't know what to modify to make the change you're suggesting.

I appreciate that a vanillla cmd.exe shell may not be sufficient to run Anaconda bits-and-pieces properly. But surely every other incantation should have worked: QtConsole, IPython Terminal, Jupyter Notebook. None of them did. Right now I have no idea what to do to get rpy2 working properly on Windows. Which is to say, I have no idea how to do whatever magic appeared to happen between originally confirming that I also had this problem (on Saturday), and then it mysteriously working OK (2 hours ago on Monday) to not working again now, after a full wipe and re-install (OK, with exception of Start Menu, which has still defeated me).

[mingwandroid]

@ijstokes look at C:\Users\ijsto\Anaconda3\cwp.py.

We can easily add $CONDA_PREFIX/Library/mingw-w64/bin to the PATH when we install Anaconda/Miniconda, but the problem with this and with trying to run things without properly activating the conda environment are:

• There is a limit to PATH length on Windows of 8192 and many applications add things there.
• DLL hell increases as things are added there (if Anaconda is installed most recently we can break programs installed earlier and vice-versa).
• Conda packages sometimes need to execute special shell scripts or batch files when an environment that they are installed into is activated. r-base on Linux and Mac are one such example. Things will fail in subtle ways due to this.

The questions that need to be addressed for this issue are:

• Do we add another entry to PATH when users select the Add to PATH option, or do we remove this option (my strong preference is to remove this option as it leads to skipping correct environment-package activation).
• Can we call activate from cwp.py instead of only adding to PATH?

[mingwandroid]

I appreciate that a vanillla cmd.exe shell may not be sufficient to run Anaconda bits-and-pieces properly. But surely every other incantation should have worked: QtConsole, IPython Terminal, Jupyter Notebook. None of them did. Right now I have no idea what to do to get rpy2 working properly on Windows. Which is to say, I have no idea how to do whatever magic appeared to happen between originally confirming that I also had this problem (on Saturday), and then it mysteriously working OK (2 hours ago on Monday) to not working again now, after a full wipe and re-install (OK, with exception of Start Menu, which has still defeated me)

For these other options to work, you need to edit cwp.pyor run from a properly activated cmd.exe e.g. Anaconda Prompt.

[ijstokes]

AFAICT the cwp.py fix above (adding an entry to the DLL directory) fixes the problem. I still have Start Menu weirdness such that my "Anaconda Prompt" isn't really (which is to say, I have to activate root before the from rpy2 import rinterface command will work), but I think that is "on my side" (but a problem I believe many will face because it seems we don't manage install/reinstall/uninstall of Start Menu entries very well). Here is a short screencast if you want to see my experience (~90 seconds):

http://screencast.com/t/FNm8EOwbZb

@mingwandroid what I still don't understand here is the origin and scope of this problem. Is it the case that today no one can use rpy2 with Windows (that is, if they have a "recent" Anaconda installation or "recent" rpy2 Conda package)? Or if it works for some people on Windows, how is it that this works? And if I'm understanding you correctly, wouldn't it be the case that this missing directory would lead to lots of things being broken (or appearing broken, which for the end user is pretty much the same thing) on Windows? But the whole issue has been very confusing for me, so I probably don't understand correctly what is going on here, including lacking a correct perspective on the scope of the problem. I do realize I am especially good at creating borked Anaconda installations or Conda environments.

[mingwandroid]

@ijstokes : everyone who runs Anaconda Prompt can use rpy2 just fine (because that runs the correct activate logic). Everyone who runs cmd.exe then activate.bat root is in the same boat (for the same reason).

The people who can't are those who use a menu shortcut (because of we need an extra PATH entry in cwp.py or fix that script to run activate) and those who elect to add Anaconda to PATH from the installer and then run things in an ad-hoc way (again because we need an extra PATH entry in the installer logic).

IMHO:

• cwp.py should call activate though instead of adding to os.environ['PATH'] directly. That way the logic is centralised and package activations get run.
• Globally adding anything to PATH is a very dodgy practice and we should remove that option and instead force everyone to flow through the correct activate logic.

As well as activate logic, there is also deactivate logic, and that is automatically performed when switching enviornments. This stuff is a fundamental part of the concept of conda environments and doing only part of the job (adding to PATH) is just asking for trouble.

[ijstokes]

@mingwandroid I'm with you. I'd even go so far as to suggest that we should do the same with Mac and Linux, so instead of adding Anaconda to PATH we have an entry source /path/to/anaconda/bin/activate root put into the user's ~/.bashrc file (or ~/.bash_profile -- whichever it is).

But it sounds like anything launched from Navigator today (in Windows) won't have a working rpy2, at least not in the root environment. Is that correct?

[mingwandroid]

That seems to be the case. I had hoped that installing jupyter into another environment then launching from that one within Navigator would cause it to activate that environment correctly but this does not seem to be the case unfortunately (@goanpeca, how do you handle this?).

We can make an update to the menuinst package quite easily that would fix this though, adding that extra path, then discuss in the weekly meeting the correct way to proceed?

[ijstokes]

@mingwandroid and @csoja I would say that it is essential 4.2.1 allows someone to conda install -c r rpy2 and for it to work properly from standard Navigator "apps" (Jupyter, QtConsole). I wouldn't want to specify the strategy for how that is achieved.

[mingwandroid]

We can roll out a new menuinst in the meantime I think?

[csoja]

yes, @mingwandroid please roll out a new menuinst.

[mingwandroid]

The new menuinst is now available, conda update menuinst should result in Navigator and Jupyter being able to import rinterface.

[aschi2]

I have this problem even after updating menuinst:

C:\Anaconda\lib\site-packages\rpy2\rinterface\__init__.py in <module>()
     85
     86 if sys.platform == 'win32':
---> 87     _load_r_dll(R_HOME)
     88
     89 # cleanup the namespace

C:\Anaconda\lib\site-packages\rpy2\rinterface\__init__.py in _load_r_dll(r_home)
     62     if r_bin not in os.environ.get('PATH'):
     63         os.environ['PATH'] = ';'.join((os.environ.get('PATH'), r_bin, r_mod))
---> 64     ctypes.CDLL(r_dll)
     65
     66

C:\Anaconda\lib\ctypes\__init__.pyc in __init__(self, name, mode, handle, use_errno, use_last_error)
    360
    361         if handle is None:
--> 362             self._handle = _dlopen(self._name, mode)
    363         else:
    364             self._handle = handle

WindowsError: [Error 126] The specified module could not be found

I installed rpy2 with conda install rpy2 if that makes a difference.

[mingwandroid]

1. How did you launch Python here? cmd.exe, Anaconda Prompt, Anaconda Navigator etc?
2. Can you show the value of os.environ['PATH'] at line 64 in C:\Anaconda\lib\site-packages\rpy2\rinterface\__init__.py please?
3. Can you show the result of conda info and conda list --show-channel-urls.

[aschi2]

1. I get the error through ipython command line and jupyter notebook.

2. Here are lines 49-64:

   def _load_r_dll(r_home):
   """Load the R.DLL matching Python's bitness.
   
   Add directories containing R DLLs to the PATH environment variable.
   
   """
   import ctypes
   r_arch = ('i386', 'x64')[sys.maxsize > 2**32]
   r_mod = os.path.join(r_home, 'modules', r_arch)
   r_bin = os.path.join(r_home, 'bin', r_arch)
   r_dll = os.path.join(r_bin, 'R.dll')
   if not os.path.exists(r_dll):
       raise RuntimeError("Unable to locate R.dll at %s" % r_dll)
   if r_bin not in os.environ.get('PATH'):
       os.environ['PATH'] = ';'.join((os.environ.get('PATH'), r_bin, r_mod))
   ctypes.CDLL(r_dll)

3. Current conda install:

          platform : win-64
     conda version : 4.3.14```

   conda is private : False conda-env version : 4.3.14 conda-build version : not installed python version : 2.7.13.final.0 requests version : 2.13.0 root environment : C:\Anaconda (writable) default environment : C:\Anaconda envs directories : C:\Anaconda\envs C:\Users*\AppData\Local\conda\conda\envs C:\Users*.conda\envs package cache : C:\Anaconda\pkgs C:\Users*\AppData\Local\conda\conda\pkgs channel URLs : https://conda.anaconda.org/anaconda-fusion/win-64 https://conda.anaconda.org/anaconda-fusion/noarch https://repo.continuum.io/pkgs/free/win-64 https://repo.continuum.io/pkgs/free/noarch https://repo.continuum.io/pkgs/r/win-64 https://repo.continuum.io/pkgs/r/noarch https://repo.continuum.io/pkgs/pro/win-64 https://repo.continuum.io/pkgs/pro/noarch https://repo.continuum.io/pkgs/msys2/win-64 https://repo.continuum.io/pkgs/msys2/noarch config file : C:\Users*.condarc offline mode : False user-agent : conda/4.3.14 requests/2.13.0 CPython/2.7.13 Windows/10 Windows/10.0.14393`

`# packages in environment at C:\Anaconda:
#
_license                  1.1                      py27_1    defaults
alabaster                 0.7.10                   py27_0    defaults
anaconda                  custom                   py27_0    defaults
anaconda-client           1.6.2                    py27_0    defaults
anaconda-navigator        1.5.0                    py27_0    defaults
anaconda-project          0.4.1                    py27_0    defaults
argcomplete               1.0.0                    py27_1    defaults
astroid                   1.4.9                    py27_0    defaults
astropy                   1.3                 np111py27_0    defaults
babel                     2.3.4                    py27_0    defaults
backports                 1.0                      py27_0    defaults
backports_abc             0.5                      py27_0    defaults
basemap                   1.0.7               np111py27_0    defaults
beautifulsoup4            4.5.3                    py27_0    defaults
bitarray                  0.8.1                    py27_1    defaults
blaze                     0.10.1                   py27_0    defaults
bleach                    1.5.0                    py27_0    defaults
bokeh                     0.12.4                   py27_0    defaults
boto                      2.45.0                   py27_0    defaults
bottleneck                1.2.0               np111py27_0    defaults
bzip2                     1.0.6                     vc9_3  [vc9]  defaults
cdecimal                  2.3                      py27_2    defaults
cffi                      1.9.1                    py27_0    defaults
chardet                   2.3.0                    py27_0    defaults
chest                     0.2.3                    py27_0    defaults
click                     6.7                      py27_0    defaults
cloudpickle               0.2.2                    py27_0    defaults
clyent                    1.2.2                    py27_0    defaults
colorama                  0.3.7                    py27_0    defaults
comtypes                  1.1.2                    py27_0    defaults
conda                     4.3.14                   py27_1    defaults
conda-env                 2.6.0                         0    defaults
configobj                 5.0.6                    py27_0    defaults
configparser              3.5.0                    py27_0    defaults
console_shortcut          0.1.1                    py27_1    defaults
contextlib2               0.5.4                    py27_0    defaults
cryptography              1.7.1                    py27_0    defaults
curl                      7.52.1                    vc9_0  [vc9]  defaults
cycler                    0.10.0                   py27_0    defaults
cython                    0.25.2                   py27_0    defaults
cytoolz                   0.8.2                    py27_0    defaults
dask                      0.14.0                   py27_0    defaults
datashape                 0.5.4                    py27_0    defaults
decorator                 4.0.11                   py27_0    defaults
dill                      0.2.5                    py27_0    defaults
docutils                  0.13.1                   py27_0    defaults
entrypoints               0.2.2                    py27_1    defaults
enum34                    1.1.6                    py27_0    defaults
et_xmlfile                1.0.1                    py27_0    defaults
fastcache                 1.0.2                    py27_1    defaults
flask                     0.12                     py27_0    defaults
flask-cors                3.0.2                    py27_0    defaults
freetype                  2.5.5                     vc9_2  [vc9]  defaults
funcsigs                  1.0.2                    py27_0    defaults
functools32               3.2.3.2                  py27_0    defaults
futures                   3.0.5                    py27_0    defaults
get_terminal_size         1.0.0                    py27_0    defaults
gevent                    1.2.1                    py27_0    defaults
greenlet                  0.4.12                   py27_0    defaults
grin                      1.2.1                    py27_3    defaults
h5py                      2.6.0               np111py27_2    defaults
hdf5                      1.8.15.1                  vc9_4  [vc9]  defaults
heapdict                  1.0.0                    py27_1    defaults
html5lib                  0.999                    py27_0    defaults
icu                       57.1                      vc9_0  [vc9]  defaults
idna                      2.2                      py27_0    defaults
imagesize                 0.7.1                    py27_0    defaults
ipaddress                 1.0.18                   py27_0    defaults
ipykernel                 4.5.2                    py27_0    defaults
ipython                   5.1.0                    py27_0    defaults
ipython_genutils          0.1.0                    py27_0    defaults
ipywidgets                5.2.2                    py27_1    defaults
isort                     4.2.5                    py27_0    defaults
itsdangerous              0.24                     py27_0    defaults
jdcal                     1.3                      py27_0    defaults
jedi                      0.9.0                    py27_1    defaults
jinja2                    2.9.5                    py27_0    defaults
jpeg                      9b                        vc9_0  [vc9]  defaults
jsonschema                2.5.1                    py27_0    defaults
jupyter                   1.0.0                    py27_3    defaults
jupyter_client            5.0.0                    py27_0    defaults
jupyter_console           5.1.0                    py27_0    defaults
jupyter_core              4.3.0                    py27_0    defaults
lazy-object-proxy         1.2.2                    py27_0    defaults
libpng                    1.6.27                    vc9_0  [vc9]  defaults
libtiff                   4.0.6                     vc9_3  [vc9]  defaults
llvmlite                  0.16.0                   py27_0    defaults
locket                    0.2.0                    py27_1    defaults
lxml                      3.7.3                    py27_0    defaults
m2w64-bwidget             1.9.10                        2    defaults
m2w64-bzip2               1.0.6                         6    defaults
m2w64-c-ares              1.11.0                        2    defaults
m2w64-ca-certificates     20150426                    103    defaults
m2w64-curl                7.48.0                        2    defaults
m2w64-expat               2.1.1                         2    defaults
m2w64-fftw                3.3.4                         6    defaults
m2w64-flac                1.3.1                         3    defaults
m2w64-gcc-libgfortran     5.3.0                         6    defaults
m2w64-gcc-libs            5.3.0                         7    defaults
m2w64-gcc-libs-core       5.3.0                         7    defaults
m2w64-gettext             0.19.7                        2    defaults
m2w64-gmp                 6.1.0                         2    defaults
m2w64-gnutls              3.4.11                        2    defaults
m2w64-gsl                 2.1                           2    defaults
m2w64-jansson             2.7                           3    defaults
m2w64-libffi              3.2.1                         4    defaults
m2w64-libiconv            1.14                          6    defaults
m2w64-libidn              1.32                          4    defaults
m2w64-libjpeg-turbo       1.4.2                         3    defaults
m2w64-libmetalink         0.1.3                         2    defaults
m2w64-libogg              1.3.2                         3    defaults
m2w64-libpng              1.6.21                        2    defaults
m2w64-libsndfile          1.0.26                        2    defaults
m2w64-libssh2             1.7.0                         2    defaults
m2w64-libsystre           1.0.1                         3    defaults
m2w64-libtasn1            4.7                           2    defaults
m2w64-libtiff             4.0.6                         2    defaults
m2w64-libtre-git          122.c2f5d13                   5    defaults
m2w64-libvorbis           1.3.5                         2    defaults
m2w64-libwinpthread-git   5.0.0.4634.697f757               2    defaults
m2w64-libxml2             2.9.3                         3    defaults
m2w64-mpfr                3.1.4                         4    defaults
m2w64-nettle              3.1.1                         4    defaults
m2w64-nghttp2             1.9.2                         2    defaults
m2w64-nlopt               2.4.2                         3    defaults
m2w64-openssl             1.0.2.g                       2    defaults
m2w64-p11-kit             0.23.2                        2    defaults
m2w64-pcre                8.38                          2    defaults
m2w64-rtmpdump-git        499.a107cef                   4    defaults
m2w64-speex               1.2rc2                        3    defaults
m2w64-speexdsp            1.2rc3                        3    defaults
m2w64-tcl                 8.6.5                         3    defaults
m2w64-tk                  8.6.5                         3    defaults
m2w64-tktable             2.10                          5    defaults
m2w64-wineditline         2.101                         5    defaults
m2w64-xz                  5.2.2                         2    defaults
m2w64-zlib                1.2.8                        10    defaults
markupsafe                0.23                     py27_2    defaults
matplotlib                2.0.0               np111py27_0    defaults
menuinst                  1.4.4                    py27_0    defaults
mistune                   0.7.3                    py27_0    defaults
mkl                       2017.0.1                      0    defaults
mkl-service               1.1.2                    py27_3    defaults
mpmath                    0.19                     py27_1    defaults
msys2-conda-epoch         20160418                      1    defaults
multipledispatch          0.4.9                    py27_0    defaults
nbconvert                 5.1.1                    py27_0    defaults
nbformat                  4.3.0                    py27_0    defaults
networkx                  1.11                     py27_0    defaults
nltk                      3.2.2                    py27_0    defaults
nose                      1.3.7                    py27_1    defaults
notebook                  4.4.1                    py27_0    defaults
numba                     0.31.0              np111py27_0    defaults
numexpr                   2.6.2               np111py27_0    defaults
numpy                     1.11.3                   py27_0    defaults
numpydoc                  0.6.0                    py27_0    defaults
oauthlib                  2.0.2                     <pip>
odo                       0.5.0                    py27_1    defaults
olefile                   0.44                     py27_0    defaults
openpyxl                  2.4.1                    py27_0    defaults
openssl                   1.0.2k                    vc9_0  [vc9]  defaults
pandas                    0.19.2              np111py27_1    defaults
pandoc                    1.19.2.1                      1    defaults
pandocfilters             1.4.1                    py27_0    defaults
partd                     0.3.7                    py27_0    defaults
path.py                   10.1                     py27_0    defaults
pathlib2                  2.2.0                    py27_0    defaults
patsy                     0.4.1                    py27_0    defaults
pep8                      1.7.0                    py27_0    defaults
pickleshare               0.7.4                    py27_0    defaults
pillow                    4.0.0                    py27_1    defaults
pip                       9.0.1                    py27_1    defaults
ply                       3.10                     py27_0    defaults
prompt_toolkit            1.0.9                    py27_0    defaults
psutil                    5.2.0                    py27_0    defaults
py                        1.4.32                   py27_0    defaults
pyasn1                    0.1.9                    py27_0    defaults
pycosat                   0.6.1                    py27_1    defaults
pycparser                 2.17                     py27_0    defaults
pycrypto                  2.6.1                    py27_5    defaults
pycurl                    7.43.0                   py27_2    defaults
pyflakes                  1.5.0                    py27_0    defaults
pygments                  2.2.0                    py27_0    defaults
pylint                    1.6.4                    py27_1    defaults
pyopenssl                 16.2.0                   py27_0    defaults
pyparsing                 2.1.4                    py27_0    defaults
pyqt                      5.6.0                    py27_2    defaults
pytables                  3.2.2               np111py27_4    defaults
pytest                    3.0.6                    py27_0    defaults
python                    2.7.13                        0    defaults
python-dateutil           2.6.0                    py27_0    defaults
pytz                      2016.10                  py27_0    defaults
pywin32                   220                      py27_2    defaults
pyyaml                    3.12                     py27_0    defaults
pyzmq                     16.0.2                   py27_0    defaults
qt                        5.6.2                     vc9_3  [vc9]  defaults
qtawesome                 0.4.4                    py27_0    defaults
qtconsole                 4.2.1                    py27_2    defaults
qtpy                      1.2.1                    py27_0    defaults
r                         3.3.2                  r3.3.2_0    defaults
r-assertthat              0.1                    r3.3.2_4    r
r-backports               1.0.4                  r3.3.2_0    r
r-base                    3.3.2                         0    defaults
r-base64enc               0.1_3                  r3.3.2_0    r
r-bh                      1.62.0_1               r3.3.2_0    r
r-bitops                  1.0_6                  r3.3.2_2    r
r-boot                    1.3_18                 r3.3.2_0    defaults
r-broom                   0.4.1                  r3.3.2_0    r
r-car                     2.1_4                  r3.3.2_0    r
r-caret                   6.0_73                 r3.3.2_0    r
r-catools                 1.17.1                 r3.3.2_2    r
r-class                   7.3_14                 r3.3.2_0    defaults
r-cluster                 2.0.5                  r3.3.2_0    defaults
r-codetools               0.2_15                 r3.3.2_0    defaults
r-colorspace              1.3_1                  r3.3.2_0    r
r-crayon                  1.3.2                  r3.3.2_0    r
r-curl                    2.3                    r3.3.2_0    r
r-data.table              1.10.0                 r3.3.2_0    r
r-dbi                     0.5_1                  r3.3.2_0    r
r-dichromat               2.0_0                  r3.3.2_2    r
r-digest                  0.6.10                 r3.3.2_0    r
r-dplyr                   0.5.0                  r3.3.2_0    r
r-essentials              1.5.2                  r3.3.2_0    r
r-evaluate                0.10                   r3.3.2_0    r
r-forcats                 0.1.1                  r3.3.2_0    r
r-foreach                 1.4.3                  r3.3.2_0    r
r-foreign                 0.8_67                 r3.3.2_0    defaults
r-formatr                 1.4                    r3.3.2_0    r
r-ggplot2                 2.2.0                  r3.3.2_0    r
r-gistr                   0.3.6                  r3.3.2_0    r
r-glmnet                  2.0_5                  r3.3.2_0    r
r-gtable                  0.2.0                  r3.3.2_0    r
r-haven                   1.0.0                  r3.3.2_0    r
r-hexbin                  1.27.1                 r3.3.2_0    r
r-highr                   0.6                    r3.3.2_0    r
r-hms                     0.3                    r3.3.2_0    r
r-htmltools               0.3.5                  r3.3.2_0    r
r-htmlwidgets             0.8                    r3.3.2_0    r
r-httpuv                  1.3.3                  r3.3.2_0    r
r-httr                    1.2.1                  r3.3.2_0    r
r-irdisplay               0.4.4                  r3.3.2_0    r
r-irkernel                0.7.1                  r3.3.2_0    r
r-iterators               1.0.8                  r3.3.2_0    r
r-jsonlite                1.1                    r3.3.2_0    r
r-kernsmooth              2.23_15                r3.3.2_0    defaults
r-knitr                   1.15.1                 r3.3.2_0    r
r-labeling                0.3                    r3.3.2_2    r
r-lattice                 0.20_34                r3.3.2_0    defaults
r-lazyeval                0.2.0                  r3.3.2_0    r
r-lme4                    1.1_12                 r3.3.2_0    r
r-lubridate               1.6.0                  r3.3.2_0    r
r-magrittr                1.5                    r3.3.2_2    r
r-maps                    3.1.1                  r3.3.2_0    r
r-markdown                0.7.7                  r3.3.2_2    r
r-mass                    7.3_45                 r3.3.2_0    defaults
r-matrix                  1.2_7.1                r3.3.2_0    defaults
r-matrixmodels            0.4_1                  r3.3.2_0    r
r-mgcv                    1.8_16                 r3.3.2_0    defaults
r-mime                    0.5                    r3.3.2_0    r
r-minqa                   1.2.4                  r3.3.2_2    r
r-mnormt                  1.5_5                  r3.3.2_0    r
r-modelmetrics            1.1.0                  r3.3.2_0    r
r-modelr                  0.1.0                  r3.3.2_0    r
r-munsell                 0.4.3                  r3.3.2_0    r
r-nlme                    3.1_128                r3.3.2_0    defaults
r-nloptr                  1.0.4                  r3.3.2_2    r
r-nnet                    7.3_12                 r3.3.2_0    defaults
r-openssl                 0.9.5                  r3.3.2_0    r
r-pbdzmq                  0.2_4                  r3.3.2_0    r
r-pbkrtest                0.4_6                  r3.3.2_0    r
r-plyr                    1.8.4                  r3.3.2_0    r
r-pryr                    0.1.2                  r3.3.2_0    r
r-psych                   1.6.9                  r3.3.2_0    r
r-purrr                   0.2.2                  r3.3.2_0    r
r-quantmod                0.4_7                  r3.3.2_0    r
r-quantreg                5.29                   r3.3.2_0    r
r-r6                      2.2.0                  r3.3.2_0    r
r-randomforest            4.6_12                 r3.3.2_0    r
r-rbokeh                  0.5.0                  r3.3.2_0    r
r-rcolorbrewer            1.1_2                  r3.3.2_3    r
r-rcpp                    0.12.8                 r3.3.2_0    r
r-rcppeigen               0.3.2.9.0              r3.3.2_0    r
r-readr                   1.0.0                  r3.3.2_0    r
r-readxl                  0.1.1                  r3.3.2_0    r
r-recommended             3.3.2                  r3.3.2_0    defaults
r-repr                    0.10                   r3.3.2_0    r
r-reshape2                1.4.2                  r3.3.2_0    r
r-rmarkdown               1.3                    r3.3.2_0    r
r-rpart                   4.1_10                 r3.3.2_0    defaults
r-rprojroot               1.1                    r3.3.2_0    r
r-rvest                   0.3.2                  r3.3.2_0    r
r-scales                  0.4.1                  r3.3.2_0    r
r-selectr                 0.3_0                  r3.3.2_0    r
r-shiny                   0.14.2                 r3.3.2_0    r
r-sourcetools             0.1.5                  r3.3.2_0    r
r-sparsem                 1.74                   r3.3.2_0    r
r-spatial                 7.3_11                 r3.3.2_0    defaults
r-stringi                 1.1.2                  r3.3.2_0    r
r-stringr                 1.1.0                  r3.3.2_0    r
r-survival                2.40_1                 r3.3.2_0    defaults
r-tibble                  1.2                    r3.3.2_0    r
r-tidyr                   0.6.0                  r3.3.2_0    r
r-tidyverse               1.0.0                  r3.3.2_0    r
r-ttr                     0.23_1                 r3.3.2_0    r
r-uuid                    0.1_2                  r3.3.2_0    r
r-xml2                    1.0.0                  r3.3.2_0    r
r-xtable                  1.8_2                  r3.3.2_0    r
r-xts                     0.9_7                  r3.3.2_2    r
r-yaml                    2.1.14                 r3.3.2_0    r
r-zoo                     1.7_13                 r3.3.2_0    r
requests                  2.13.0                   py27_0    defaults
requests-cache            0.4.12                   py27_0    conda-forge
requests-oauthlib         0.8.0                     <pip>
rope                      0.9.4                    py27_1    defaults
rpy2                      2.8.5              py27r3.3.2_2    defaults
ruamel_yaml               0.11.14                  py27_1    defaults
scandir                   1.5                      py27_0    defaults
scikit-image              0.12.3              np111py27_1    defaults
scikit-learn              0.18.1              np111py27_1    defaults
scipy                     0.18.1              np111py27_1    defaults
seaborn                   0.7.1                    py27_0    defaults
setuptools                27.2.0                   py27_1    defaults
simplegeneric             0.8.1                    py27_1    defaults
singledispatch            3.4.0.3                  py27_0    defaults
sip                       4.18                     py27_0    defaults
six                       1.10.0                   py27_0    defaults
snowballstemmer           1.2.1                    py27_0    defaults
sockjs-tornado            1.0.3                    py27_0    defaults
sphinx                    1.5.1                    py27_0    defaults
spyder                    3.1.3                    py27_0    defaults
sqlalchemy                1.1.6                    py27_0    defaults
ssl_match_hostname        3.4.0.2                  py27_1    defaults
statsmodels               0.8.0               np111py27_0    defaults
subprocess32              3.2.7                    py27_0    defaults
sympy                     1.0                      py27_0    defaults
testpath                  0.3                      py27_0    defaults
tk                        8.5.18                    vc9_0  [vc9]  defaults
toolz                     0.8.2                    py27_0    defaults
tornado                   4.4.2                    py27_0    defaults
traitlets                 4.3.2                    py27_0    defaults
tweepy                    3.5.0                     <pip>
unicodecsv                0.14.1                   py27_0    defaults
vs2008_runtime            9.00.30729.5054               0    defaults
wcwidth                   0.1.7                    py27_0    defaults
werkzeug                  0.11.15                  py27_0    defaults
wheel                     0.29.0                   py27_0    defaults
widgetsnbextension        1.2.6                    py27_0    defaults
win_unicode_console       0.5                      py27_0    defaults
wrapt                     1.10.8                   py27_0    defaults
xlrd                      1.0.0                    py27_0    defaults
xlsxwriter                0.9.6                    py27_0    defaults
xlwings                   0.10.2                   py27_0    defaults
xlwt                      1.2.0                    py27_0    defaults
zlib                      1.2.8                     vc9_3  [vc9]  defaults
`

[mingwandroid]

1. How did you launch the ipython command line and jupyter notebook?
2. I wanted to know the value of os.environ['PATH'] at that location, not what the code there is. ipdb or pdb can be used to get this. If you can't give that please show the value of echo %PATH% from the command line that you would use to launch ipython.

[aschi2]

1. cmd.exe for both cases
2. echo %PATH% yields: C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;C:\Anaconda;C:\Anaconda\Scripts;C:\Anaconda\Library\bin;C:\Users\*\AppData\Local\Microsoft\WindowsApps;C:\Users\*\AppData\Local\rodeo\app-2.5.2\bin;C:\Program Files (x86)\Microsoft VS Code\bin

Sorry I don't know how to use ipdb or pdb.

[mingwandroid]

Your PATH is incorrect. My recommendations, in order (pick one) are:

1. Remove Anaconda entires from your PATH altogether and then always launch Anaconda Prompt and/or use the other Anaconda Start Menu entries.
2. Fix the Anaconda entries in your PATH. Your PATH should be:

   C:\Anaconda;C:\Anaconda\Library\mingw-w64\bin;C:\Anaconda\Library\usr\bin;C:\Anaconda\Library\bin;C:\Anaconda\Scripts;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;C:\Users\*\AppData\Local\Microsoft\WindowsApps;C:\Users\*\AppData\Local\rodeo\app-2.5.2\bin;C:\Program Files (x86)\Microsoft VS Code\bin

Out of interest, how did you set PATH? Did you do it manually recently? Or is this from an old installation done some time ago?

[aschi2]

It's been a while so my memory is a little hazy. I had anaconda installed for a couple years, but was having problems updating packages, so I uninstalled and reinstalled it. I believe I manually changed the path when I first installed it, but I can't remember for sure. I did not manually change path the second time I installed it.

[mingwandroid]

Ok let me know if my suggestions work for you.

[aschi2]

I edited my path to what you posted and it worked! Thanks!

[mingwandroid]

No problem. Thanks for letting me know.

[schlichtanders]

we just mangaged to get rid of the error on Windows 10 64bit, by doing the following:

• conda install m2w64-toolchain
• adding C:\Anaconda3\Library\mingw-w64\bin to your system PATH

Explanation: as Windows throws OSError: [WinError 126] The specified module could not be found also if other dll dependencies are not found, this probably adds the decisive dll's to the path.

How have we found it: This package is a conda dependency for theano to compile stuff for C on 64 bit Windows. In fact it worked.

Question: Maybe this package should also be added as a dependency to the anaconda package rpy2?

Enjoy

[mingwandroid]

m2w64-toolchain is not a dependency of rpy2. Some of its dependencies are shared with those of rpy2 but installing rpy2 on its own should work just fine as it correctly lists all of its own dependencies:

conda install rpy2

The following NEW packages will be INSTALLED:

    m2w64-bwidget:           1.9.10-2
    m2w64-bzip2:             1.0.6-6
    m2w64-expat:             2.1.1-2
    m2w64-fftw:              3.3.4-6
    m2w64-flac:              1.3.1-3
    m2w64-gcc-libgfortran:   5.3.0-6
    m2w64-gcc-libs:          5.3.0-7
    m2w64-gcc-libs-core:     5.3.0-7
    m2w64-gettext:           0.19.7-2
    m2w64-gmp:               6.1.0-2
    m2w64-gsl:               2.1-2
    m2w64-libiconv:          1.14-6
    m2w64-libjpeg-turbo:     1.4.2-3
    m2w64-libogg:            1.3.2-3
    m2w64-libpng:            1.6.21-2
    m2w64-libsndfile:        1.0.26-2
    m2w64-libtiff:           4.0.6-2
    m2w64-libvorbis:         1.3.5-2
    m2w64-libwinpthread-git: 5.0.0.4634.697f757-2
    m2w64-libxml2:           2.9.3-3
    m2w64-mpfr:              3.1.4-4
    m2w64-pcre:              8.38-2
    m2w64-speex:             1.2rc2-3
    m2w64-speexdsp:          1.2rc3-3
    m2w64-tcl:               8.6.5-3
    m2w64-tk:                8.6.5-3
    m2w64-tktable:           2.10-5
    m2w64-wineditline:       2.101-5
    m2w64-xz:                5.2.2-2
    m2w64-zlib:              1.2.8-10
    r:                       3.3.2-r3.3.2_0
    r-base:                  3.3.2-0
    r-boot:                  1.3_18-r3.3.2_0
    r-class:                 7.3_14-r3.3.2_0
    r-cluster:               2.0.5-r3.3.2_0
    r-codetools:             0.2_15-r3.3.2_0
    r-foreign:               0.8_67-r3.3.2_0
    r-kernsmooth:            2.23_15-r3.3.2_0
    r-lattice:               0.20_34-r3.3.2_0
    r-mass:                  7.3_45-r3.3.2_0
    r-matrix:                1.2_7.1-r3.3.2_0
    r-mgcv:                  1.8_16-r3.3.2_0
    r-nlme:                  3.1_128-r3.3.2_0
    r-nnet:                  7.3_12-r3.3.2_0
    r-recommended:           3.3.2-r3.3.2_0
    r-rpart:                 4.1_10-r3.3.2_0
    r-spatial:               7.3_11-r3.3.2_0
    r-survival:              2.40_1-r3.3.2_0
    rpy2:                    2.8.5-py27r3.3.2_2
    singledispatch:          3.4.0.3-py27_0

^ here, the m2w64 are the ones that are in the mingw-w64\bin folder that are needed. m2w64-toolchain is the GCC compiler which rpy2 does not use.

The thing that fixed your installation is modifying your PATH.

I am guessing you have an old installation of Anaconda and you allowed it to modify your system PATH at install time? I encourage not doing that in general and instead using the Anaconda Prompt or else Anaconda Navigator to launch your Anaconda software. This will set PATH correctly for that session only without globally modifying your system. Alternatively, if you reinstall from a more recent installer you can let that set your system PATH (with the downside of risking breaking other software and/or not working properly).

[santhosh77h]

@mingwandroid in windows 7 64 bit this issue solved with suggestions of @schlichtanders

[mingwandroid]

@santhosh77h I'm not sure what you mean, as I explained in some detail what @schlichtanders got wrong in https://github.com/ContinuumIO/anaconda-issues/issues/1129#issuecomment-289413749

You are best off using the Anaconda Prompt, not modifying your PATH and there is absolutely no need to install m2w64-toolchain, but I'm glad you got it working, albeit as side effects of incorrect instructions.

[ChandelierX]

I just faced the same problem and I, I have to say, really suffered from it these two days. I use Anaconda3-4.3.1 on 64-bit Windows 7, and install rpy2 in version 2.8.5 from the way mentioned in https://anaconda.org/r/rpy2. Nonetheless, pycharm and ipython etc. always says OSError: [WinError 126] The specified module could not be found. To begin with, I tried the way mentioned by @schlichtanders , unfortunately it proved not to work on my PC. When I was about to give up using rpy2, I tried @mingwandroid 's way, thank God it works! To convey my gratitude I register this account. Finally Thanks a lot!

[ChandelierX]

Oh there's One more thing to mention. Coincidentally, when I used former version of anaconda, everything about rpy2 went well, except its "plot" function as well as that my Pycharm kept indexing until my memory space were not enough. Therefore I reinstalled Pycharm and Anaconda (New version, and during the this process I definitely chose the option--set anaconda in environment variable, actually I clicked all options in case). After that, my rpy2 began to be problematic in finding module. As a result, I guess this problem may have something to do with the new version of Anaconda. @mingwandroid

[mingwandroid]

I don't recommend allowing the Anaconda installer to set your PATH. It's very problematic on Windows due to DLL hell and it also avoids proper environment activation (which the Anaconda Prompt and Anaconda Navigator will do for you. Otherwise you should use cmd.exe then run call C:\Users\me\Anaconda\Scripts\activate, then, to launch PyCharm with the correct environment you should then run it manually using the full path of pycharm.exe (with double-quotes around spaces in e.g. C:\"Program Files"\...)

[mingwandroid]

The 'PyCharm indexing forever' thing is well known across all the intellij products and I would love to see it get fixed.

To make this less of a problem when using conda, I can recommend installing Miniconda instead of Anaconda and then only installing the packages you need.

[ChandelierX]

Thanks a lot for your kindness and the extra advices about miniconda and proper enviroment activation. Fortunately, I checked my anaconda environment, and I seem not encounter enviroment activation problems now, since I am in the only 'root environment', and I don't want to reinstall any conda again until necessary ...... Do you know: Does proper environment activation always happen in my case? so that I'd better to launch Pycharm in the way you described each time when I create another environment?

[mingwandroid]

Env activation can involve more than setting PATH (batch files or shell scripts can be run). There are only 3 ways on Windows to activate an env properly which I listed above. Yes, PyCharm should be launched from the properly activated env. if you expect it to work correctly (including the root env.).

[Stephanie-Ting-3]

I've tried all this from Cygwin and none of it worked, but the problem went away when I started Jupyter Notebook from Anaconda Prompt.

[mingwandroid]

At present we don't support Cygwin (or the MSYS2/Git for Windows shells, PowerShell or Linux Subsystem for Windows) though we do plan to support as many of these as we can, provided they don't incur too much maintenance cost.

[ChandelierX]

Weird, I tried your advice by activating my new environment and then launched my pycharm in the same anaconda prompt, however in Pycharm, I was still in the original environment and was able to use rpy2. Only after I manually changed my pycharm--setting--project_interpreter from 'Anaconda3\python.exe' to 'Anaconda3\envs\tensorflow\python.exe', I could shift to the new environment in pycharm... I guess it is just OK, right? Since nothing seems to go wrong now? Again, thank you ^v^.

[mingwandroid]

Yes, PyCharm also has its own idea about the Python that is active, so you need to line up both the conda environment (via Scripts/activate) and the PyCharm project interpreter for things to work properly.

This is particularly important if you use any of our software that uses GCC-on-Windows (mingw-w64), for example RPy2 and XGBoost.

[ChandelierX]

I see it! It's Kind of you~

[sulaimanvesal]

I would like to confirm that @schlichtanders method worked for me pretty good. Windows 10, Python 3.5. I was struggling with this issue for several days. Thank you.