search

conda-forge / openblas-feedstock

A conda-smithy repository for openblas.
BSD 3-Clause "New" or "Revised" License
9 stars 39 forks source link

Deleted old builds? #10

Closed cpaulik closed 8 years ago

cpaulik commented 8 years ago

When I try to install one of my environment.yml files I get

Error: Package missing in current linux-64 channels: 
  - openblas 0.2.18 1

The reason seems to be that this build was removed from https://anaconda.org/conda-forge/openblas/files

This already happened with gdal a while ago https://github.com/conda-forge/gdal-feedstock/issues/78

Is this common practice with conda-forge builds? I'd just like to know because this makes them essentially useless for making environment.yml files.

ocefpaf commented 8 years ago

As a scientist I am 100% with you on pining things down to the build number to ensure reproducible envs. As a packager that is bad practice, specially b/c there is no guarantee that the build number did not change (see the hotfix discussion https://github.com/conda-forge/conda-forge.github.io/pull/170).

To answer you question:

Is this common practice with conda-forge builds?

Sadly yes need b/c we to get rid of bad builds to circumvent a conda bug that installs a lower build number even when a higher one is present.

I'd just like to know because this makes them essentially useless for making environment.yml files.

It makes it hard to pin down to the build number, but you can still have environment.yml and, in theory, you would get the latest (and corrected) build number.

PS: I am not a maintainer here and I don't know the all the reasons for the deleted build number. (I believe it was the libgcc dependency on Linux causing trouble down the road.) So I will wait for a maintainer to clarify the reason and close the issue. However, let me know if your ideas about reproducible envs as I am an advocate to keep the bad builds even when we know they are a fatal error. (We just need to fix conda first :smile: to avoid breaking everything though.)

cpaulik commented 8 years ago

Thanks. Ok this means that conda needs to be improved to really enable reproducible builds.

This is probably the wrong thread but recently I noticed a few things that make it hard to work with conda in this context:

  1. Different conda versions have different package resolution strategies. Because of this it can happen that an environment.yml file stops working with a newer conda version. This would be no big problem except for:
  2. It seems that older conda versions can not be used after approx. 1 year since the package format is still changing. See e.g. https://github.com/conda/conda/issues/1642 . But this might only apply to new packages. So the conda version should also be fixed in an environment.yml if 'old' conda versions can work with 'old' packages.
  3. Conda does not allow (without manual editing) to create an environment.yml without build numbers AFAIK. This could be an option to get the right version at least.

Probably an issue with these points in the conda repository would make sense.

ocefpaf commented 8 years ago

This is probably the wrong thread but recently I noticed a few things that make it hard to work with conda in this context

It is :smile:

But I would like to discuss that more with you as I am also interested in that problem. Please let's open a new issue and discuss it there.

jakirkham commented 8 years ago

I agree with your concern @cpaulik about deleting old packages. Hence I raised this issue ( https://github.com/conda-forge/conda-forge.github.io/issues/181 ).

Though @ocefpaf is of course correct that we have discussed hot-fixing packages. The intent with the latter is more about fixing pinnings to ensure reproducibility of a working environment. However, there is always a possibility that something like that goes wrong. In any event, it seems too controversial ATM for hot-fixing to go anywhere yet.

jakirkham commented 8 years ago

Thanks @cpaulik for raising this. Going to close this out as the discussion seems to have moved to conda.