force conda lsst to build already built packages

[danielsf]

I would like to try to add a command that forces conda-lsst to build packages, even if they have already been built and uploaded to eupsforge. The use case is as follows:

Since we are currently doing an end-run around conda's poor version resolution by blowing away every past version whenever we upload a new version to eupsforge, I would like to be able to

1) create a fresh clone of conda-lsst on the build machine 2) build the latest version in its entirety (even packages that have already been built and uploaded to eupsforge) 3) delete the old version on eupsforge in its entirety 4) upload from the build machine to eupsforge

This avoids the ~ hour or so of work involved with uploading the new version to eupsforge and then selectively deleting out-of-date packages from the dev channel.

[mjuric]

Two ways to get that now:

Option 1: after the build is done, run this one liner:

cat recipes/rebuild.sh  | grep ^rebuild | awk '{ print $2"-"$3".tar.bz2" } > packages.txt

to get the list of packages belonging to the build you just did. Then run something like mv $(cat packages.txt) ../tmp; rm *; mv ../tmp/* .; conda index . to leave only those files on the channel. This shouldn't take more than a minute or two. I think you could fully automate this with a few lines of bash.

Option 2: every time you build a new release, a) clean up the local build directory (usually, e.g., ./miniconda/conda-bld/osx-64), and then build into a new, empty, conda channel (e.g., instead of dev, build into dev2 -- change your entry in ~/.condarc and create the ~mjuric/public_html/conda/dev2/osx-64 directory on lsst-dev). Once the build is done (and uploaded), just rename the dev2 directory to dev.

Would either of these solve your problem?

[danielsf]

These would probably solve the problem. It's just that I'd rather avoid all of the channel switching involved in Option 2 and I don't speak fluent enough bash to understand exactly what is going on in Option 1. I'm willing to do whatever work is involved in my suggestion. I just want to see if it is possible.

[mjuric]

Closing as I think this is out of scope for the recipe maker & better handled by an external script.