search

conda-forge / conda-forge.github.io

The conda-forge website.
https://conda-forge.org
BSD 3-Clause "New" or "Revised" License
128 stars 274 forks source link

(Many?) Packages include pip-installed requirements due to buggy conda-build <3.17 #684

Open tjd2002 opened 5 years ago

tjd2002 commented 5 years ago

As of ~October 2018, the conda-forge docs started recommending that recipes remove the --no-deps flag from their pip install lines (#653 #659), presumably because conda-build started setting an env var (PIP_NO_DEPENDENCIES) to make this the default behavior.

Unfortunately, there was a bug in conda-build at that time (https://github.com/conda/conda-build/issues/3254), which inverted the sense of the flag. The bug was introduced in conda/conda-build#3053 and first shipped in release 3.12.1; it was fixed by https://github.com/conda/conda-build/pull/3271, shipped in release 3.17.0 .

This means that any packages built during this time that did not explicitly specify the --no-deps flag will have been built with all their requirements included in the package! See https://github.com/conda-forge/bleach-feedstock/issues/17 for an example of what that looks like. Needless to say, this is very bad; leads to overwriting of Conda-installed packages with arbitrary/old versions, etc.

Now that Conda-forge appears to be building with a fixed conda-build=3.17.4, the solution is very simple: bump the build number for each package and rebuild.

After encountering this in bleach, I started poking around and quickly found more packages with the problem unfixed (I issued PRs): https://github.com/conda-forge/pyexcel-ods3-feedstock/pull/3 https://github.com/conda-forge/apache-libcloud-feedstock/pull/16

I also found several cases of packagers independently finding the issue in the last few weeks, and fixing it by re-adding the --no-deps flag: https://github.com/conda-forge/numcodecs-feedstock/pull/32 https://github.com/conda-forge/aubio-feedstock/pull/16 https://github.com/conda-forge/orange3-feedstock/pull/54 https://github.com/conda-forge/pymultinest-feedstock/issues/4

I suspect there are quite a lot more packages affected; I stopped searching after a while to write this issue. I think it would make sense to run a batch bump-and-rebuild on any package that may be affected. I'm not sure what the best way would be to identify such packages, or how to run such a batch job.

I also wonder if there could be some non-repairable damage done to folks installs by these packages. If 'bleach' was bringing along its own copy of six and webencodings, and then I upgrade bleach to a version that no longer includes those packages, is conda capable of restoring those packages to their previously-installed versions? Or do I need to recreate that conda env?

cc @minrk @jakirkham

nicoddemus commented 5 years ago

then I upgrade bleach to a version that no longer includes those packages, is conda capable of restoring those packages to their previously-installed versions? Or do I need to recreate that conda env?

What happened for us is that conda would remove the six.py file when updating bleach b0 to b1, leaving the environment in a broken state. Our solution was to recreate the environment by deleting it and issuing conda devenv again.

tjd2002 commented 5 years ago

Bummer. Sounds like this means anyone who has ever installed any of these broken packages may end up with a broken env once the packages are fixed.

Does this mean we should be careful about pushing the fixed packages??

nicoddemus commented 5 years ago

Bummer

Yes, but no biggie. The hard part was to track this problem down (thanks @prusse-martin!). Now we have re-provisioned all of our CI workers so things should be back to normal: we have checked and it seems all the other packages in our stack don't have this problem.

Does this mean we should be careful about pushing the fixed packages??

Not sure, depends on what it means "to be careful". Preferably it should be done in one fell swoop, rather than small increments of fixed packages: each new fixed package has the potential to break existing environments.

tjd2002 commented 5 years ago

Over at https://github.com/conda-forge/bleach-feedstock/pull/18#issuecomment-447062555 @mariusvniekerk wrote:

So one useful source of data for determining which artifacts are affect is regro/libcfgraph which has file listings for every conda forge artifact