search

DeviationTX / deviation

Custom firmware for RC Transmitters
http://www.deviationtx.com
GNU General Public License v3.0
251 stars 155 forks source link

Improve Travis build coverage #577

Open PhracturedBlue opened 5 years ago

PhracturedBlue commented 5 years ago

Currently our target matrix for travis does not cover all of the targets in the nightly build. Additionally, the travis build takes almost twice as long to run the matrix as the nightly build (which has more targets). The primary reason for the matrix build was to reduce runtime, and we seem to be failing at that. We need to rethink the travis build setup

howard0su commented 5 years ago

I actual don't mind to have a longer runtime for travis. Normally my project in the work take about 45minutes to finish. I would prefer to add all targets into travis-ci.

PhracturedBlue commented 5 years ago

because I like to have a quick response to a change if it is broken

TheRealMoeder commented 5 years ago

Matrix is faster, since usually, at least four builds are running in parallel and each is less than two minutes. Nightly takes over nine minutes until we get first results.

TheRealMoeder commented 5 years ago

Check PR #578

PhracturedBlue commented 5 years ago

I'm not sure how travis reports build time. I would expect that the nightly build will fail really quickly if we break devo8 or devo10 since they build 1st, so even the monolithic build may be faster than the matrix build in the fast-fail as well as the full-run case. And with a monolthic build we could probably increase parallelism and see further benefit (I'm not sure how travis allocates resources, so maybe not). On the other hand, the logs from monolithic builds are more difficult to parse.

When I've worked with gitlab or Jenkins CI, I am able to setup sophisticated multi-stage builds where you can have a common setup and then parallelize just the build stage. and treat each part as its own stage. This could be helpful, because we do not really benefit from having separate VMs for each matrix target, and the setup time is significant.

howard0su commented 5 years ago

Did u try this? https://docs.travis-ci.com/user/build-stages If not I can take a look today.

TheRealMoeder commented 5 years ago

From what I've read, each build stage job is run in a separate VM, so this wouldn't give any gain.

howard0su commented 5 years ago

my idea is first stage just build some basic targets like what we do today. second stage build zips which run a full build. so if it breaks the common targets, you will notice immediately.

On Sun, Jan 13, 2019 at 5:09 PM TheRealMoeder notifications@github.com wrote:

From what I've read, each build stage job is run in a separate VM, so this wouldn't give any gain.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/DeviationTX/deviation/issues/577#issuecomment-453813651, or mute the thread https://github.com/notifications/unsubscribe-auth/AAuzRL8ahRaWvPn9nVoj7cjx98VNj69gks5vCvfXgaJpZM4Z8u2m .

-- -Howard

howard0su commented 5 years ago

basically fail fast approach.

On Sun, Jan 13, 2019 at 5:23 PM Howard Su howard0su@gmail.com wrote:

my idea is first stage just build some basic targets like what we do today. second stage build zips which run a full build. so if it breaks the common targets, you will notice immediately.

On Sun, Jan 13, 2019 at 5:09 PM TheRealMoeder notifications@github.com wrote:

From what I've read, each build stage job is run in a separate VM, so this wouldn't give any gain.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/DeviationTX/deviation/issues/577#issuecomment-453813651, or mute the thread https://github.com/notifications/unsubscribe-auth/AAuzRL8ahRaWvPn9nVoj7cjx98VNj69gks5vCvfXgaJpZM4Z8u2m .

-- -Howard

-- -Howard

TheRealMoeder commented 5 years ago

How is this going to be faster than current approach with (expanded) matrix? Currently we have first results for major targets within a minute. Also, we have small logs and at a glance you can see which target fails.

howard0su commented 5 years ago

Current approach, we need go to click the link to travis website to find out when the job is running. the status only reports back when the job finish. I just want something which can fail if some early targets already failed before long running job (make zips).

TheRealMoeder commented 5 years ago

If I'm not mistaken, if one of the Travis matrix jobs fails, we already get the red x in GitHub for the commit. Build time is around 20 seconds per target, about 40 seconds are needed for setting up the VM before the build begins. So having more than 3 builds in each Travis job will not be anymore efficient. I guess one option would be to make matrix of 5 for the larger targets (each major transmitter/emu line: black and white, color, text, emu, winemu) where zips for all "subtargets" are build and the build script posting a GitHub status back for each individual target.

howard0su commented 5 years ago

if that's the case, my suggestion doesn't matter.

I think your suggestion works. maybe you want another target other than zips.

On Sun, Jan 13, 2019 at 7:38 PM TheRealMoeder notifications@github.com wrote:

If I'm not mistaken, if one of the Travis matrix jobs fails, we already get the red x in GitHub for the commit. Build time is around 20 seconds per target, about 40 seconds are needed for setting up the VM before the build begins. So having more than 3 builds in each Travis job will not be anymore efficient. I guess one option would be to make matrix of 5 for the larger targets (each major transmitter/emu line: black and white, color, text, emu, wine) where zips for all "subtargets" are build and the build script posting a GitHub status back for each individual target.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/DeviationTX/deviation/issues/577#issuecomment-453822392, or mute the thread https://github.com/notifications/unsubscribe-auth/AAuzRKMSCun8KBOhh9mpvNNEFjF9gquvks5vCxqjgaJpZM4Z8u2m .

-- -Howard

howard0su commented 5 years ago

shall we at least add at9, devof4 into the build matrix?

PhracturedBlue commented 5 years ago

at9 should already be there today. I'm not opposed to adding f4

howard0su commented 5 years ago

yes. why at9 is like TARGET= but others are MAKETARGET. what's the difference?

On Fri, Jan 18, 2019 at 8:12 AM PhracturedBlue notifications@github.com wrote:

at9 should already be there today. I'm not opposed to adding f4

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/DeviationTX/deviation/issues/577#issuecomment-455379767, or mute the thread https://github.com/notifications/unsubscribe-auth/AAuzRAvH93bHf76izi4eQTRKkdPHBjTbks5vERF9gaJpZM4Z8u2m .

-- -Howard

PhracturedBlue commented 5 years ago

there shouldn't be any. probably just added incorrectly