Replace Travis CI with GitHub Actions

[pkuczynski]

GitHub Actions is much faster and responsive than Travis

[pkuczynski]

@eregon do you maybe know why install phase for truffleruby is so ridiculously slow?

[eregon]

@pkuczynski It seems on TravisCI it was using the Bundler cache: https://travis-ci.org/railsconfig/config/jobs/633397197

I wonder how long it is on TravisCI without cache and if it's any better or worse in GitHub Actions.

It is slow indeed though, we're looking at what's so slow, it seems to have regressed (https://github.com/oracle/truffleruby/issues/1398).

[eregon]

Ah but that was a run on the sinatra Gemfile, it's not really comparable.

[pkuczynski]

@eregon it runs on a few gemfiles, ranging from rails 4.2 - 5.2 as well as sinatra. Still very slow to bundle install, especially compared to mri ruby. Not sure why would that be the case :/

[eregon]

It's a combination of things and currently I think partly some inefficiencies in the zlib C-extension when running on TruffleRuby. https://github.com/oracle/truffleruby/issues/1398 has a bit more details. We plan to look at this soon and profile it in GitHub Actions to see what's the bottleneck in that context.

[pkuczynski]

After many fail and error I finally was able to implement caching (it's quite compliacted due using appraisal and testing against extensive ruby/rails matrix), but that didn't helped much:

Before:

* 2.4: 4m 12s
* 2.5: 3m 22s
* 2.6: 3m 25s
* 2.7: 3m 50s
* truffleruby: 42m 30s

After:

* 2.4: 2m 14s
* 2.5: 2m 32s
* 2.6: 2m 11s
* 2.7: 2m 20s
* truffleruby: 34m 41s

I am not sure why after restoring the cache it does not pick up all installed gems, but only some and still needs to install some others. It does not happen on my local machine... You can observe thus behavior here: https://github.com/railsconfig/config/runs/379685645

[eregon]

@pkuczynski I would guess the reason is the Bundler vendor path is not picked for bundle exec appraisal install. I think setting BUNDLE_PATH globally for the whole job might help.

[pkuczynski]

Shouldn't bundle config path solve it? That's the recommendation from actions/cache documentation...

I also added JRuby to the test matrix https://github.com/rubyconfig/config/pull/228 (much easier doable than in Travis). It took only 6m to run the pipeline (2x slower than mri ruby). Check it out in https://github.com/rubyconfig/config/runs/379884681. In this light >30m for truffleruby is really sad :(

[eregon]

Shouldn't bundle config path solve it? That's the recommendation from actions/cache documentation...

You're right, that should be fine since that's global and saved in ~/.bundle/config.

The issue is then the path is relative to where bundle install is run. I wonder if it would work with an absolute path like bundle config path $HOME/.bundler_cache. Then it might help to cache not just one Gemfile but all of them.

[pkuczynski]

I am now setting BUNDLE_PATH and that helps a lot with reusing dependencies:

https://github.com/rubyconfig/config/pull/228/files#diff-353995fb5bebaef307ec25fa5be39042R17

Now truffleruby is 10m instead of 45m :) Still way slower then mri (45-70s)

[pkuczynski]

Btw. jruby is lesst then 5m

You can see all results here https://github.com/rubyconfig/config/pull/228/checks?check_run_id=384222383

[pkuczynski]

For comparison on travis testing only sinatra was taking about 15m. I could never get rails examples working with truffleruby, but now I am testing all apps in a single test run, which seems to be way more efficient than spinning multiple builds.

For comparison: https://travis-ci.org/railsconfig/config/builds/634314479

[eregon]

@pkuczynski Very happy to hear you found a way to cache Bundler properly :)

It's expected more optimizing implementations are slower to run tests/specs, because tests/specs typically do everything but only a few times, nothing in a loop and nothing to really optimize. More optimizing implementations profile the execution, which has some cost in interpreter. TruffleRuby doesn't optimize for interpreter performance but for performance after JIT compilation. Trade-offs essentially.

I'm sure we can always improve, but those times don't seem unreasonable to me. Bundler performance is much slower than it should though as I said earlier, we're investigating that.

[pkuczynski]

Sure thing! Thanks for the explanation. I just wanted to follow up with feedback and observations :) Now when the cashing works, build seems more reasonable :)