Closed MilesCranmer closed 1 year ago
Patch coverage has no change and project coverage change: +0.19
:tada:
Comparison is base (
bcaba00
) 67.03% compared to head (e6f3744
) 67.22%.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.
I'm not sure I'm following. All PyCall.jl wants to do is ensure that the conda environment it is operating in has numpy
installed. Normally, this is a conda environment that is created specifically for a Julia install, but from a conda installed Julia this is an existing conda environment.
Could we consider adding --freeze-installed
instead?
Wait, I thought this line was the cause of the downstream issues? Doesn’t Conda.add()
update the numpy version and reinstall?
https://docs.conda.io/projects/conda/en/latest/commands/install.html
Conda attempts to install the newest versions of the requested packages. To accomplish this, it may update some packages that are already installed, or install additional packages. To prevent existing packages from updating, use the --freeze-installed option.
(maybe that’s what you meant by --freeze-installed
?)
I think it’s safer to entirely turn off calls to conda install
when inside a conda-forge build script though. Just feels a bit dangerous. What if even with freeze-installed
, it installs some other packages which are not present in the env that conda-forge is building?
Actually, I think we want --satisfied-skip-solve
What if even with
freeze-installed
, it installs some other packages which are not present in the env that conda-forge is building?
It should throw errors since you did not previously add all the needed dependencies.
Okay I added the required functionality in https://github.com/JuliaPy/Conda.jl/pull/240
I think it’s safer to entirely turn off calls to
conda install
when inside a conda-forge build script though.
If this is possible, it is desirable. However, it likely should be done as a patch in conda-forge, not here. I will leave that up to you though. Why would PyCall need numpy to begin with? That’s kinda odd and arbitrary (isn’t it supposed to be a generic python caller?)
Why would PyCall need numpy to begin with? That’s kinda odd and arbitrary (isn’t it supposed to be a generic python caller?)
In order to wrap Julia arrays with numpy arrays when calling Python.
Why would PyCall need numpy to begin with? That’s kinda odd and arbitrary (isn’t it supposed to be a generic python caller?)
In order to wrap Julia arrays with numpy arrays when calling Python.
Why would PyCall need numpy to begin with? That’s kinda odd and arbitrary (isn’t it supposed to be a generic python caller?)
In order to wrap Julia arrays with numpy arrays when calling Python.
@mkitti Updated with satisfied_skip_solve=true
@stevengj could we merge and release this change?
Just pinging this - could this be merged @stevengj?
I only have an auto-merge button here.
An chance we could get the other CI bits to pass? Python 2.7 should probably just be retired...
It says:
Error: Version 2.7 with arch x64 not found
so we can't even test it. Also, pyjulia already dropped support.
@stevengj sorry for the spam but we really need this fixed to solve some downstream failures. Any chance you could merge+release?
@MilesCranmer, is pycall available in conda-forge somehow? What in conda-forge uses it? If that's the case, we could patch it and move the ball forward instead of waiting.
Or is it the thing that Julia calls and we bundle in your pysr setup?
If so, we could point your pysr package on Conda-forge to use this https://github.com/MilesCranmer/PyCall.jl/tree/patch-numpy-install (I don't know how that's done in Julia, but I assume it is easily doable, right?)
Right, we bundle it with the PySR install as a special julia depot.
Right now PyCall.jl automatically installs numpy into a conda environment. This install cannot be disabled. If this occurs from within an existing conda environment, it can overwrite the numpy installed by a conda-forge build. This violates the assumptions of conda-forge, and can silently corrupt any conda env that depends on numpy.
This issue was noticed in https://github.com/conda-forge/pysr-feedstock/pull/85, manifesting in this issue: https://github.com/conda-forge/scipy-feedstock/issues/238. We believe this is caused by the line:
https://github.com/JuliaPy/PyCall.jl/blob/bcaba00d1e2c412b2f61d33343ef5a9ab1b9258a/deps/build.jl#L79
This PR enables the
PYCALL_INSTALL_NUMPY
environment flag which can be set tofalse
to disable theConda.add("numpy")
line and prevent the corruption of a conda env. The default behavior is unchanged if this flag is not set, so this is backwards compatible.cc @h-vetinari @mkitti @ngam