search

crystal-lang / distribution-scripts

40 stars 24 forks source link

Refactor darwin build job #186

Open straight-shoota opened 2 years ago

straight-shoota commented 2 years ago

darwin release builds take a long time to finish. The dist_darwin job on circle ci usually takes ~50 minutes for releaes builds. The comparable dist_linux job finishes in about 15 minutes (example: 1.4.0 release workflow).

We're using omnibus for dist_darwin. It appears to be building all libraries from scratch. The first 10+ minutes are entirely spend just downloading stuff. This seems to be very inefficient. omnibus itself draws in some dependencies and then the entire process is very slow.

We actually just need to install a couple of libraries and then build the compiler. That's a pretty straighforward task and IMO shouldn't require omnibus. I think we could achieve this with a couple of make recipes for building the dependencies (and ideally cache them to avoid rebuilding every time). Maybe an even better alternative would be to build in the nix environment which we already use for the test jobs on GitHub actions.

/cc @bcardiff @matiasgarciaisaia @beta-ziliani WDYT?

beta-ziliani commented 2 years ago

IIUC omnibus is used for (potential?) darwin specific stuff. But I guess we can switch to nix and switch back when we see a real need for it.

matiasgarciaisaia commented 2 years ago

Replacing omnibus with another thing may work just fine - the only caveat would be to pay close attention to compatibility issues with different macOS versions (I guess we don't support OpenDarwin or distros like that).

straight-shoota commented 1 year ago

FTR: The latest build times on macOS are more like 36 minutes, so that's an improvement of the initial reporting. I have no idea why, though. I guess that isn't too bad actually, considering the multiarch process means the compiler is built twice.

Still, our omnibus setup has shown to be outdated and hard to work with. I'd appreciate migrating to a better solution.

straight-shoota commented 1 year ago

Maybe a "better solution" could also mean updating omnibus to a more recent version and partially improving the build configuration.

One thing I noticed is that omnibus actually provides a caching mechanism but I don't think we're using it. At least not in CI. It might just work locally if you keep the cache directory. @bcardiff Do you remember why we're not synchronizing the build cache across CI runs and instead jerry-rig a cache with the llvm_bin artifact on S3? https://github.com/chef/omnibus/blob/main/docs/Build%20Cache.md

bcardiff commented 1 year ago

I don't recall anything regarding omnibus caching. So it's worth a try.

I think there was some reason why we didn't update to omnibus, but I don't remember why. I'll if I find something, ultimately it was a long time ago and hopefully things changed in the right direction.

straight-shoota commented 1 year ago

This might also be relevant: #134