search

flatironinstitute / jax-finufft

JAX bindings to the Flatiron Institute Non-uniform Fast Fourier Transform (FINUFFT) library
Apache License 2.0
80 stars 3 forks source link

Add Jenkins CI #31

Closed lgarrison closed 1 year ago

lgarrison commented 1 year ago

This adds a Jenkins CI config to run the GPU (and CPU) tests on the FI Jenkins platform. The test are currently passing and the output can be seen here: https://jenkins.flatironinstitute.org/job/jax-finufft/

There's a few TODOs, like figuring out the right number of OpenMP threads and getting the GitHub webhooks installed, but overall I think this is pretty close to done!

dfm commented 1 year ago

This looks great! Let me know what web hook to install and I'll get that set up.

One note: looking through the logs I see:

WARNING: jax 0.4.19 does not provide the extra 'cuda12-local'

and it also looks like the most recent build is reusing an old environment since dependencies like jax are already installed. Is that expected or something to worry about?

lgarrison commented 1 year ago

That's a good catch about jax already being installed, the storage must be persisting in some way. I'll take a look. (It also explains why the Ninja build is only building 8 targets.)

The cuda12-local thing is weird, I think it only started happening in the last few weeks. My suspicion is that this is a pip bug/new behavior of some kind that's mapping the underscore to the dash, or that there's some chain of dependencies where one of the dependencies is accidentally using a dash instead of an underscore. But ultimately it seems to be installing the right thing.

lgarrison commented 1 year ago

Dylan helped me fix the persistent storage issue (Jenkins tries to save the workspace so you don't have to do a git checkout from scratch each time, but there's an option to clean the workspace before running). Seems to be behaving correctly now.

There are two more automation steps to take care of: 1) To get status updates from Jenkins, we need to add @flatiron-jenkins to the repo with write access 2) To trigger Jenkins builds automatically, we need to add the webhook. For security reasons, only Dylan should do this (the webhook has a secret key), so we can either add @dylex as an admin to this repo (even temporarily), or transfer the repo to the flatironinstitute GitHub org. Either one is fine, let me know what you'd like to do @dfm!

dfm commented 1 year ago

Great! It doesn't seem to be possible to add admins on a personal repo (it's only possible on an organization repo), so I'm happy to transfer this to flatironinstitute.