treeform / puppy

Puppy fetches via HTTP and HTTPS
MIT License
184 stars 27 forks source link

Make CI faster #99

Open iffy opened 1 year ago

iffy commented 1 year ago

I'm coming here from what @guzba mentioned here about slow macOS tests: https://github.com/treeform/pixie/pull/510

This PR makes CI faster for macOS. Here's a comparison:

Currently This PR
Total time 15-16min 8-9min
Cumulative CPU time 1h 14m 50s 21m 54s
Usage report Latest PR

Granted, there seems to be a flakey test that fails sometimes on macOS, so maybe some of the speedup comes from skipped tests? But if you compare a single macOS test run, you can see the savings during the install step latest 14:39 run vs this PR 0:48 run

I'm not offended if you don't want the PR -- close it if you don't. I just hate to see slow CI builds :)

treeform commented 1 year ago

This could be pretty cool if it works!

1.2.2 does not supply a binary?

iffy commented 1 year ago

1.2.2 does not supply a binary?

Unfortunately, binaries that work with macOS weren't produced until v1.2.6 AFAICT. If you do find a nightly URL that works for 1.2.2, you can specify it as described here: https://github.com/iffy/install-nim/blob/master/install-nim.sh#L24

As-is, this PR uses choosenim to install 1.2.2.

amkrajewski commented 4 months ago

Have you tried installing using Homebrew? I found it to behave as expected and during my testing workflow runs it installs nim (v2.0.0) in just about 5s on macos-14 runners or 11s on macos-12 runners.

In my specific case, is also appreciably faster than other runners. Installing (1) nim, (2) Arraymancer and wimpy (through nimble), (3) compiling my code, and (4) running all tests takes under 60s.

To be a bit more quantitative:

Runner Install Tool Install Time Test Runtime
macos-14 Homebrew 5s 16s
ubuntu-latest jiro4989 7s 30s
macos-12 Homebrew 11s 40s
windows-latest Conda+jiro4989 185s 51s

There is no magic involved. Here is the job from my workflow YAML file:

test-M1:
    runs-on: macos-14
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Install Nim with Homebrew
        run: brew install nim
      - name: Install dependencies
        run: nimble install -y arraymancer nimpy
      - name: Compile nimplex
        run: nim c -d:release nimplex.nim
      - name: Run grid tests
        run: nim c -r -d:release tests/runAll.nim

I hope it helps :)