search

regro / cf-scripts

Flagship repo for cf-regro-autotick-bot
Other
53 stars 74 forks source link

Removing yum_requirements.txt when possible #358

Open jakirkham opened 6 years ago

jakirkham commented 6 years ago

Historically yum_requirements.txt has been used as a kludge to add dependencies to the Linux build environment that we don't have packages for in conda-forge or defaults. Many of these were added long ago. However as conda-forge and defaults have grown the landscape has changed. We no longer need to add a Fortran compiler this way ( https://github.com/regro/cf-scripts/issues/357 ). Also things like ruby and various build tools have been added as conda-forge packages.

Given this, it would be very useful to have a migrator that identifies things that no longer need to be yum installed and attempts to remove them in favor of conda-forge packages instead. This would remove lines in yum_requirements.txt (or the whole file if empty), add these packages as host and run dependencies, re-render to update the CI files to avoid yum installing the unneeded packages (or yum altogether if possible), and bump the build number. May require a mapping between yum and conda-forge/defaults names for packages.

This would go a long ways toward eliminating our dependence on system package managers. As a bonus, this would mitigate the effect of issue ( https://github.com/conda-forge/conda-smithy/issues/890 ).

CJ-Wright commented 6 years ago

This would be great! We might be able to implement a mapping to handle both use cases

  1. If the requirement is not needed remove it.
  2. If the requirement has a conda-forge equivalent swap it out for a host/build requirement.
CJ-Wright commented 5 years ago

To my mind this would entail:

  1. During the make graph section, try to also get the yum_requirements.txt file from the feedstock.
  2. Creating a new migrator which sees if any of the deps in yum_requirements.text is available from CF, remove it and put it into the recipe.
  3. repeat 2 for all things in the yum file
  4. If yum file is empty delete it.

Bonus points if we can print out all the things we are missing from CF that are in yum files.

jakirkham commented 5 years ago

So one thing to consider is whether we get the package from conda-forge/defaults or whether we use a CDT. cc-ing a couple people that may have thoughts on this.

cc @pkgw @mingwandroid

mingwandroid commented 5 years ago

IMHO CDTs are always preferable to system packages because they guarantee centos6 compatibility and have been proven to work well.

Using our own packages is generally preferable to using CDTs too.

pkgw commented 5 years ago

And especially to maintain parity between Defaults and conda-forge, CDTs definitely seem like the route that we should encourage people to use.