Consider a new CI tool to replace TravisCI

[kapoorlakshya]

Summary

TravisCI recently announced public projects will now have limited build time (10K credits / 1000 mins on Linux) per month. The average CI time for this gem on Travis has been ~20 mins, so under the new OSS plan, we'll run out in roughly 50 jobs on Linux. This number is actually lower when we factor in macOS jobs which have a higher credit utilization @ 50/min (equals 200 mins total) vs 10/min for Linux.

However, OSS projects can request a higher credit allowance via email, but that adds on a layer of admin work to manage comms with Travis + manage credit utilization that one of us will have to be responsible for. The alternative is to research other options and migrate our CI pipeline.

Anyway, this may not end up being a huge issue for us given the low release frequency, but is just something to consider.

Potential alternatives that support Linux, macOS, and Windows

The following seem to have a more OSS-friendly (free) plan based on my limited research:

• Appveyor
  • Pros
  • Unlimited build time
  • Low effort migration since we're already using it
  • Cons
  • No concurrency (limited to 1)
• GitHub Actions:
  • Pros
  • Unlimited build time for public repos (pricing)
  • concurrency - 20 total, 5 on macOS
  • Can replace current Appveyor pipeline and consolidate into one
  • Cons
  • Will have to build from scratch; possibly longer migration process

Out of the above options, I'm leaning toward GitHub Actions. @titusfortner @twalpole Any strong preferences or suggestions?

[titusfortner]

Yeah, Selenium has moved to GitHub Actions. I've never used it, but I think that's the way we should go.

[kapoorlakshya]

Sounds good! I'll look into replacing Travis (and Appveyor if possible) with GitHub Actions.

[kapoorlakshya]

Have a branch with this implemented, but the macOS run for msedgedriver is failing because of an error that only happens for one spec:

Spec: rspec ./spec/webdrivers/edgedriver_spec.rb:103 # Webdrivers::Edgedriver#update when correct binary is not found downloads binary Error:

dyld: Library not loaded: @rpath/libc++.dylib
  Referenced from: /Users/runner/.webdrivers/msedgedriver
  Reason: image not found

Unable to reproduce this locally and all of the solutions I found via Google have failed so far.

[kapoorlakshya]

Reported above error and found a temporary work around for it - https://github.com/actions/virtual-environments/issues/2462

Successful run - https://github.com/kapoorlakshya/webdrivers/runs/1716241451?check_suite_focus=true

[kapoorlakshya]

Turns out the issue is with the driver. It needs the libc++.dylib that exists in the zip for some versions, but not all. All details are documented in my comment here to close https://github.com/actions/virtual-environments/issues/2462.

Now working on updating this gem to also extract that lib, if it exists, along with msedgedriver.

[kapoorlakshya]

Just noticed that the upcoming v88 and v89 do not need or include the libc++.dylib... so maybe this lib is a temporary requirement for v86 and v87 (?). And since no one here has reported this causing an issue, it might not be a common issue amongst our users or not many people are actively using this driver. Not sure.

Anyway, I am going to hold off on making any changes to support this since it appears to be either temporary or intermittent depending on the driver release. I have a work around in place to make the driver work in the macOS image on GitHub Actions. Will consider implementing a permanent solution if future releases permanently or intermittently require it, or if someone reports it as an issue here.

Closed by 7754cd9347a26aabf25dc90d7ba9608ce1a501d4.

[kapoorlakshya]

@titusfortner Can you please disable the required checks on Appveyor and TravisCI, and enable all the ones on GitHub Actions. I'll delete the config files once this change is made. Thanks!