search

sandialabs / pyGSTi

A python implementation of Gate Set Tomography
http://www.pygsti.info
Apache License 2.0
134 stars 56 forks source link

Idea: Switch macOS tests to run against beta branch #387

Closed coreyostrove closed 8 months ago

coreyostrove commented 9 months ago

Recently it has been taking forever (hours) for the standard unit test suite to get picked up by the macOS github runners. We also recently learned that the macOS runners use up our free computational credits at 10X the linux rate and 5X the Windows rate. Both of these things are kind of annoying. Question for discussion:

Should we change our CI workflow so that the tests in unit only get run on the macOS runners upon pushing to beta (i.e. when merged into develop), rather than against all pushes to any branch?

sserita commented 9 months ago

I would be for this. The Mac tests rarely fail due to OS specific issues, so only checking on beta seems like a completely good solution to me.

A follow-up idea: We sometimes but rarely have Python version problems. Thoughts on only testing the lowest and highest versions of Python we support on most branches, and then testing all versions on either develop/beta or just beta?

coreyostrove commented 8 months ago

Testing only the newest and oldest supported versions of python for most branches sounds good to me as well. I'll add these changes to my to-do list and update the workflows.

sserita commented 8 months ago

Closed as of #388.

kgantchev commented 8 months ago

@coreyostrove Hi, I'm the CEO and co-founder of FlyCI. If you're concerned about your GitHub runners eating up your free tier minutes, then definitely try the FlyCI's M1 and M2 runners which are on average about 2x faster and 2x cheaper than GitHub's runners. We also offer a free tier of 500 mins/month on the M1 runner so you don't have to use up your GitHub actions free tier minutes.

Easily replace your M1 runners: 

jobs:
 ci:
-    runs-on: macos-latest
+    runs-on: flyci-macos-large-latest-m1
   steps:
   - name: 👀 Checkout repo
     uses: actions/checkout@v4

Exclusive M2 runners: 

jobs:
  ci:
-    runs-on: macos-latest
+    runs-on: flyci-macos-large-latest-m2
    steps:
      - name: 👀 Checkout repo
        uses: actions/checkout@v4

We'll be very happy to have you on board.