MacPython / openblas-libs

BSD 2-Clause "Simplified" License
9 stars 13 forks source link

Nightly linux builds #103

Closed honno closed 1 year ago

honno commented 1 year ago

Resolves https://github.com/MacPython/openblas-libs/issues/100.

As suggested by @MattiP, I've tackled this by having the nightly builds be uploaded to the same filenames with "HEAD" where the version would be. This is a crude approach compared to how other nightlies work (see scipy-wheels-nightly), but should be fine for the purposes of testing NumPy and SciPy against OpenBLAS nightlies.

Example filenames ``` openblas-HEAD-musllinux_1_1_x86_64.tar.gz openblas-HEAD-manylinux2014_x86_64.tar.gz openblas64_-HEAD-musllinux_1_1_x86_64.tar.gz openblas64_-HEAD-manylinux2014_x86_64.tar.gz openblas64_-HEAD-manylinux2010_x86_64.tar.gz openblas-HEAD-manylinux2014_i686.tar.gz openblas-HEAD-manylinux2010_x86_64.tar.gz openblas-HEAD-manylinux2010_i686.tar.gz ```

This PR is only generating linux builds (basically copy-pasted the multibuild.yml workflow). Some linux builds fail, which I can have a look at but might be a cakewalk to resolve for you folks :sweat_smile: As the new workflow runs on cron rather than push, you can look at https://github.com/honno/openblas-libs/pull/2 (test PR I made). From the latest Actions run, the following jobs are currently failing:

nightly multibuild / build (ubuntu-latest, x64, i686, 1)
nightly multibuild / build (ubuntu-latest, x64, i686, 1, 2010)
nightly multibuild / build (ubuntu-latest, x64, i686, 1, 2014)

Also note I removed Mac and Windows builds just so I could push a MVP on nightlies out the door, but I could re-instate Mac builds at least as they already work with multibuild.yml and thus should with nightly_multibuild.yml (although when I tried before they failed too).

cc @steppi

honno commented 1 year ago

This PR now starts from https://github.com/MacPython/openblas-libs/pull/104, as it'll need it anyway to work.

With an eye to merging this with the other build, we could do something like this to identify a cron run or a manually-triggered CI run

env:
  NIGHTLY: ${{ github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' }}

and use that to override other settings like the tarball name and OPENBLAS_COMMIT

Couldn't you use OPENBLAS_COMMIT=HEAD or OPENBLAS_COMMIT=develop instead? Then rather than add a NIGHTLY env variable, you could check $OPENBLAS_COMMIT == "HEAD".

So I've mangled nightly builds into multibuild.yml, using your suggested method of defining a NIGHTLY env. It seems to work nicely enough, but I don't have a sense for these things.

Unfortunately I couldn't figure out why bash scripts wouldn't recognise env variables on GitHub Actions, where they would when run locally. The pass through of NIGHTLY is a bit noisy (i.e. https://github.com/MacPython/openblas-libs/pull/103#discussion_r1252717202 seems necessary as dcker_build_wrap.sh is used in build_steps.sh::build_lib), so did want to just check the env variable directly when defining versions in build_steps.sh::do_build_lib instead:

-     local version=$(cd OpenBLAS && git describe --tags --abbrev=8)
+     if [ "$NIGHTLY" = "true" ]; then
+         local version="HEAD"
+     else
+         local version=$(cd OpenBLAS && git describe --tags --abbrev=8)
+     fi
mattip commented 1 year ago

Let's give it a shot. I can do a manual workflow run once this is merged.