Reduce CI jobs - Githubissues

RevenueCat / purchases-ios

In-app purchases and subscriptions made easy. Support for iOS, watchOS, tvOS, macOS, and visionOS.

https://www.revenuecat.com/

MIT License

2.22k stars 295 forks source link

Reduce CI jobs #4025

Open aboedo opened 1 week ago

aboedo commented 1 week ago

Summary

Will reduce the number of CI jobs we're running so that we:

run all jobs on release PRs
have some trigger to manually run all non-essential jobs based on something (could be a comment, a tag, a webhook)
then on regular commits run only essential jobs

Details:

Creates a new "run-manual-tests" workflow, that can be triggered through circleCI UI. I'll add a separate github action as a follow-up PR so that we can also trigger it via a comment in a PR.
Specifies the Xcode version explicitly for jobs in the job definition. I saw that we were always specifying it when calling the job, which would lead to some duplication, and in some cases it made the job by itself not make a lot of sense - if a job is called "run tests in xcode 14", the xcode version shouldn't really be a param
Adds a new job pod-lib-lint, which calls Cocoapods's pod lib lint. This is a very helpful job in that it runs checks on all deployment targets / OS versions / platforms, so it can catch a lot of wrong availability checks that we might miss. It was already a part of some of the other workflows, so this just makes it more explicit and removes it from the places where it was left redundant.
Also adds installation tests as part of the checks we run so that we catch some other error early. These were previously only used in the deploy workflow.

aboedo commented 1 week ago

another obvious benefit: much faster testing time for regular commits

vegaro commented 1 week ago

Heads up I just merged https://github.com/RevenueCat/purchases-ios/pull/4029 with a small change to the config.yml

aboedo commented 4 days ago

I'm going to try running the rest of the jobs manually to ensure that the system is working as expected, but at least regular jobs are running just fine

aboedo commented 4 days ago

ok, all jobs other than docs-build work correctly and @fire-at-will mentioned earlier today that that one was broken, so I imagine it's not related to the changes here.

I haven't gotten the branch filters to work well yet, though, so I'll figure that out and then ask for re-review

fire-at-will commented 4 days ago

@aboedo I'm seeing docs-build passing in my PRs right now, it's docs-deploy that failed in the 5.1.0 release pipeline yesterday

aboedo commented 4 days ago

@fire-at-will thanks for the heads up! I'll check it out then

aboedo commented 4 days ago

@fire-at-will I had the wrong xcode version on it! I'm glad you brought it up. I had to go one by one on the jobs and copy the xcode version, I guess I had a copy/paste issue on that one

aboedo commented 4 days ago

Okay, I think this is good to re-review now. A lot has changed since the original approach, but I did test that:

the jobs will correctly trigger from a release PR (see #4050)
the jobs will correctly trigger from a manual action in circleci
they run correctly once they do trigger, meaning that the refactors for xcode versions are correct

aboedo commented 2 hours ago

Not sure how you were planning to implement that but I was thinking maybe we could have something in our CircleCI orb that allows us to establish certain comment to workflow triggers, that way we can reuse it in the other repos in the future as needed. Not needed for this PR of course

Yeah, that'd be great. I've kinda struggled with it a bit because it's hard to test and it gets triggered directly from the main branch, but I'd be happy to update it it afterwards and make it generic to all repos if we find a good use for it @tonidero

aboedo commented 2 hours ago

@tonidero @vegaro @joshdholtz comments addressed! Thanks for the thorough reviews!