pip not installed along with Conda / PyCall

[jwscook]

Hi Devs,

I'm coming up against a problem running my tests on Appveyor. See this CI job as an example failure.

I've got a project with a few python dependencies that are both packaged e.g. scipy and non-packaged (a repo on github). To install thems in the build.jl I use commands like:

Using Conda
pip = joinpath(Conda.BINDIR, "pip")
run(`$pip install scipy`)
run(`$pip install git+https://github.com/ScottishCovidResponse/data_pipeline_api@0.7.0`)

which works well on travis but on Appveyor errors out because it can't find pip

┌ Error: Error building `SimulationData`: 
│    Building Conda ─→ `C:\Users\appveyor\.julia\packages\Conda\3rPhK\deps\build.log`
│    Building PyCall → `C:\Users\appveyor\.julia\packages\PyCall\zqDXB\deps\build.log`
│ ERROR: LoadError: IOError: could not spawn `'C:\Users\appveyor\.julia\conda\3\Library\bin\pip' install scipy pyyaml setuptools-scm aiohttp`: no such file or directory (ENOENT)
│ Stacktrace:
│  [1] _spawn_primitive(::String, ::Cmd, ::Array{Any,1}) at .\process.jl:99
│  [2] #550 at .\process.jl:112 [inlined]
│  [3] setup_stdios(::Base.var"#550#551"{Cmd}, ::Array{Any,1}) at .\process.jl:196
│  [4] _spawn at .\process.jl:111 [inlined]
│  [5] run(::Cmd; wait::Bool) at .\process.jl:439
│  [6] run(::Cmd) at .\process.jl:438
│  [7] top-level scope at C:\projects\simulationdata-jl\deps\build.jl:8
│  [8] include(::String) at .\client.jl:439
│  [9] top-level scope at none:5
│ in expression starting at C:\projects\simulationdata-jl\deps\build.jl:8
└ @ Pkg.Operations D:\buildbot\worker\package_win32\build\usr\share\julia\stdlib\v1.4\Pkg\src\Operations.jl:899
""

I've thought about using Conda to install https://github.com/ScottishCovidResponse/data_pipeline_api@0.7.0 in isolation of pip but can't make that work either.

Any pointers for putting pip in C:\Users\appveyor\.julia\conda\3\Library\bin\?

Thanks, James

[simonbyrne]

I'm not 100% sure: is pip an executable? Do you need to call python pip instead?

A better solution might be https://github.com/JuliaPy/Conda.jl/pull/171: maybe chime in there, or see if you can reuse some of that PR?

(also, you should be able to install scipy via Conda).

[jwscook]

Pip is an executable, but I believe you can also call it via python pip -m .... Conda can install all of the dependencies that have been packaged e.g. scipy, but I couldn't get it to install https://github.com/ScottishCovidResponse/data_pipeline_api@0.7.0. I'll try over at Conda.jl.

[simonbyrne]

What I meant is that I think pip is a Python script: on Unix you can just set the executable permission and use a shebang, but this doesn't work in Windows (only binaries and batch files are directly executable).

[jwscook]

I tried downloading and installing via get-pip.py and it produced a file called Pip.exe so I assume it's literally an executable rather than a file masquerading as one. I'm a bit confused why it's possible to call it as a python module though python -m pip ....