pyimport_conda not finding the package?

[ChrisRackauckas]

MWE:

using PyCall
ENV["PYTHON"] = ""
using Pkg; Pkg.build("PyCall")
pyimport_conda("scipy.optimize", "scipy")

Outputs:

 Conda ─→ `C:\Users\accou\.julia\packages\Conda\x5ml4\deps\build.log`
   Building PyCall → `C:\Users\accou\.julia\packages\PyCall\tqyST\deps\build.log`   
[ Info: Installing scipy.optimize via the Conda scipy package...
[ Info: Running `conda install -y scipy` in root environment
Collecting package metadata (current_repodata.json): done
Solving environment: done

# All requested packages already installed.

PyError (PyImport_ImportModule

The Python package scipy.optimize could not be imported by pyimport. Usually this means
that you did not install scipy.optimize in the Python version being used by PyCall.

PyCall is currently configured to use the Julia-specific Python distribution
installed by the Conda.jl package.  To install the scipy.optimize module, you can
use `pyimport_conda("scipy.optimize", PKG)`, where PKG is the Anaconda
package the contains the module scipy.optimize, or alternatively you can use the
Conda package directly (via `using Conda` followed by `Conda.add` etcetera).

Alternatively, if you want to use a different Python distribution on your
system, such as a system-wide Python (as opposed to the Julia-specific Python),
you can re-configure PyCall with that Python.   As explained in the PyCall
documentation, set ENV["PYTHON"] to the path/name of the python executable
you want to use, run Pkg.build("PyCall"), and re-launch Julia.

) <class 'ImportError'>
ImportError('DLL load failed while importing _iterative: The specified module could not be found.')
  File "C:\Users\accou\.julia\conda\3\lib\site-packages\scipy\optimize\__init__.py", line 400, in <module>
    from .optimize import *
  File "C:\Users\accou\.julia\conda\3\lib\site-packages\scipy\optimize\optimize.py", line 36, in <module>
    from ._numdiff import approx_derivative
  File "C:\Users\accou\.julia\conda\3\lib\site-packages\scipy\optimize\_numdiff.py", line 6, in <module>
    from scipy.sparse.linalg import LinearOperator
  File "C:\Users\accou\.julia\conda\3\lib\site-packages\scipy\sparse\linalg\__init__.py", line 111, in <module>
    from .isolve import *
  File "C:\Users\accou\.julia\conda\3\lib\site-packages\scipy\sparse\linalg\isolve\__init__.py", line 4, in <module>
    from .iterative import *
  File "C:\Users\accou\.julia\conda\3\lib\site-packages\scipy\sparse\linalg\isolve\iterative.py", line 8, in <module>
    from . import _iterative

pyimport(::String) at PyCall.jl:547
pyimport_conda(::String, ::String, ::String) at PyCall.jl:711
pyimport_conda(::String, ::String) at PyCall.jl:704
top-level scope at test.jl:103

[ChrisRackauckas]

This is odd because this was working for years and I'm not quite sure how or what to debug here.

[ChrisRackauckas]

I went into the Conda.jl folder and directly ran that Python and got:

Python 3.8.5 (default, Sep  3 2020, 21:29:08) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32

Warning:
This Python interpreter is in a conda environment, but the environment has
not been activated.  Libraries may fail to load.  To activate this environment
please see https://conda.io/activation

Type "help", "copyright", "credits" or "license" for more information.
>>> import scipy
C:\Users\accou\.julia\conda\3\lib\site-packages\numpy\__init__.py:138: UserWarning: mkl-service package failed to import, therefore Intel(R) MKL initialization ensuring its correct out-of-the box operation under condition when Gnu OpenMP had already been loaded by Python process is not assured. Please install mkl-service package, see http://github.com/IntelPython/mkl-service
  from . import _distributor_init
Traceback (most recent call last):
  File "C:\Users\accou\.julia\conda\3\lib\site-packages\numpy\core\__init__.py", line 22, in <module>
    from . import multiarray
  File "C:\Users\accou\.julia\conda\3\lib\site-packages\numpy\core\multiarray.py", line 12, in <module>
    from . import overrides
  File "C:\Users\accou\.julia\conda\3\lib\site-packages\numpy\core\overrides.py", line 7, in <module>
    from numpy.core._multiarray_umath import (
ImportError: DLL load failed while importing _multiarray_umath: The specified module could not be found.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\accou\.julia\conda\3\lib\site-packages\scipy\__init__.py", line 61, in <module>
    from numpy import show_config as show_numpy_config
  File "C:\Users\accou\.julia\conda\3\lib\site-packages\numpy\__init__.py", line 140, in <module>
    from . import core
  File "C:\Users\accou\.julia\conda\3\lib\site-packages\numpy\core\__init__.py", line 48, in <module>
    raise ImportError(msg)
ImportError:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.8 from "C:\Users\accou\.julia\conda\3\python.exe"
  * The NumPy version is: "1.19.2"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: DLL load failed while importing _multiarray_umath: The specified module could not be found.

[rkurchin]

This is the same error I was getting with the Windows CI on my package that kept failing. Are you running on Windows? If so, do you see the same issue on other platforms?