Save build artifacts in CI

maxitg commented 3 years ago

Changes

Circle CI now saves the built libraries (including the metadata JSON) and the paclet file as artifacts.

Comments

Note that the paclet file only has the Linux library. We will need to either cross-compile or run jobs on Mac and Windows to get a multi-platform paclet file, which is the goal.
If we can achieve that, we will no longer need to rely on internal Wolfram infrastructure for building (which is inaccessible to external developers).

Examples

Here they are: https://app.circleci.com/pipelines/github/maxitg/SetReplace/1292/workflows/bea84003-2279-43e9-8b84-87d013e6d532/jobs/1683/artifacts

This change is

maxitg commented 3 years ago

@taliesinb, that will definitely be useful, but it will be difficult to include them into the CI builds because Circle CI does not seem to support ARM yet: https://discuss.circleci.com/t/arm-builds/29102. Perhaps we can cross-compile them from Linux.

maxitg commented 3 years ago

The goal is to have the builds on macOS and Windows as well. Then, we can create the multi-platform paclets suitable for releases completely on Circle CI, which means we won't have to rely on internal Wolfram infrastructure, which is suboptimal because:

Only internal people have access.
Even I don't have access to changing the build config, which means doing something as simple as changing Xcode or WL version might take 100+ hours.
It is congested, sometimes it takes a significant amount of time for the build to even start. It's gotten better recently after updating the Xcode version (see the previous item), but on Circle CI the builds always start instantly.
It is manual, one has to connect to the VPN, push a specific branch to stash, and start the build, whereas Circle CI will run automatically for all commits on all branches, and will immediately tell us if the builds on other platforms fail.

maxitg commented 3 years ago

And also, once Circle CI can create paclets, we can implement a CD as well, where the paclets will just get deployed automatically after each new build to master without any manual involvement. We can even set it up in a way that it auto-updates users' paclets without any work from our side except for merging a branch to master.

daneelsan commented 3 years ago

I see, that seems really convenient. I once wanted to build a paclet for macos and didn't know how to do that.

maxitg / SetReplace