search

conda-forge / conda-smithy

The tool for managing conda-forge feedstocks.
https://conda-forge.org/
BSD 3-Clause "New" or "Revised" License
151 stars 180 forks source link

Using multistage pipelines (particularly with split packages ;) #1151

Open jakirkham opened 5 years ago

jakirkham commented 5 years ago

Azure has been advertising their multistage pipelines over the past few months, it could be interesting for us to explore this feature. In particular this could be really handy with split packages. For example a package that builds a C/C++ library as one package and then build Python bindings to that library. In these cases we might benefit from one job for the C/C++ library and multiple jobs for the Python bindings (dependent on the original single C/C++ job).

cc @mariusvniekerk (in case this is of interest 😉)

mariusvniekerk commented 5 years ago

Main trick is just keeping track of state (aka the things that have been built)

saraedum commented 5 years ago

@jakirkham so you envision having several recipes in one feedstock that depend on each other? Or would you somehow want to use multiple outputs of one recipe still?

jakirkham commented 4 years ago

It would still have multiple outputs.

An example of this would be a feedstock with one output myclib and another output python-myclib, which builds against all supported python versions. Here we would want to build myclib once and then reuse that when building python-myclib for each python version.

Today we rebuild myclib for every python version as well.