Continuous Integration improvements - (or 'do something about Travis CI')

[yeastplume]

The current situation with Travis CI is becoming more unworkable on each commit. Though our tests could probably be sped up in a few places, they're basically fine and run in very reasonable amounts of time when done locally... on TravisCI, however, they're extremely inconsistent and error during build more often than not. This is due to a few factors (like the build taking a lot of time and resources due to certain dependencies, as well as doing a full build of dependencies for each create) but our build and tests really aren't something so out of the ordinary that a run-of-the-mill CI system shouldn't be able to handle it.

Some options might be (with the caveat that I haven't spent more time thinking about this than it's taken to write the issue):

-Investigate build and travis CI setup to see if there's anything obvious we might be able to do to speed up builds -Investigate other possible build systems -Investigate any optimisations we can do to tests (though it seems in most cases a non-responsive build is the issue, so I'd leave this as a last resort).

[bladedoyle]

Travis can be configured to allow a longer period of "no output" before timeout: https://docs.travis-ci.com/user/common-build-problems/#My-builds-are-timing-out "travis_wait".

But in my experience travis is just very flaky and can not give consistent results. I have found Jenkins with declarative pipelines to be a good alternative.

[ignopeverell]

Btw, likely still not enough, but I've commented out the right-now-useless code coverage during the week-end, leading to a 30% test speed improvement.

Someone know of an already dockerized build/test system? The less configuration the better, so anyone of us could set it up in 15 min on any machine or VM of our choosing.

[sesam]

ping @hendi or if there's a savior server angel in the house?

On Mar 5, 2018 7:39 PM, "Ignotus Peverell" notifications@github.com wrote:

Btw, likely still not enough, but I've commented out the right-now-useless code coverage during the week-end, leading to a 30% test speed improvement.

Someone know of an already dockerized build/test system? The less configuration the better, so anyone of us could set it up in 15 min on any machine or VM of our choosing.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/mimblewimble/grin/issues/749#issuecomment-370519298, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAi2S0fsddLrjCWRVDPsilNjvRtc_gTks5tbYZ6gaJpZM4ScdY- .

[hendi]

The GNOME project's librsvg uses Rust and moved to GitLab recently and had some success there. Their gitlab-ci.yml looks straigt forward: https://gitlab.gnome.org/GNOME/librsvg/blob/master/.gitlab-ci.yml

I can definitely whip up a server for running CI tasks. I've heard that it's easy to add build agents to GitLab, so maybe mirroring this repo to GitLab and then have they run the CI process and fan out to a server of ours would work.

I'm also looking into Buildbot and sccache in order to build a simple yet fast CI config for grin.

[antiochp]

gitlab might be worth a try. :+1: on exploring that for the CI functionality. It's not amazing as a full github replacement but possible to run CI there with the main repo still in github.

I'm wondering if it may be worth exploring using gitlab to build a docker image at build time that we can then use to run the tests (not 100% sure you can do this easily with rust though as different tests build individual binaries).

One of the things that feels really frustrating in travis is we build the grin project from scratch in every job. There's no easy way to build it once and then use the same build for each test job. There is no real pipeline functionality in TravisCI where the build job triggers downstream test jobs. So we spend ~20mins building grin to then run the grin_util tests and the tests themselves only take a couple of seconds. Ideally we build once (even a slow build for all test binaries), then reuse this across multiple fast parallel test jobs.

[hendi]

Oh okay, I had that solved in one of my playgrounds: build grin once (slow), then fan out and run several tests (fast). I wasn't satisfied by that and wanted to look into making the build faster, i.e. incremental compilation.

[sesam]

It seems prudent to try both paths; both try the setup @hendi has, and have a gitlab account that tracks the master branch. But if we want gitlab:s CI for our PRs, then we'd probably need to somehow get every PR added as a branch on gitlab?

[0xmichalis]

Yeah, you would need to push to a branch in gitlab, open merge requests, and sync the commit status back to github.

Both Gitlab and Github use webhooks so there should be no considerable lag between the two. https://docs.gitlab.com/ce/user/project/integrations/webhooks.html https://developer.github.com/webhooks/

The real problem is how does the bot choose to sync PRs on (re)start? You would probably need to use a naming convention in MR titles. Also how does it rectify failed pushes to the gitlab fork? Again, probably (ab)use the MR title to track pushed Github HEADs.

[yeastplume]

closing for now