Suggestion: Setup travis-ci for repository

[stopspazzing]

Just a suggestion, could you setup your repository with https://travis-ci.org/ so builds can be checked? Could also have it auto upload the files created for nighly builds. Just a suggestion. Doesnt cost anything.

[nworbnhoj]

Thanks - good suggestion - awesome tool.

Except I have not managed to get it to work yet - do you have any experience?

There are a variety of limits to navigate due to the size of the Gargoyle build. The automated build fails if: console output exceeds about 6M (assumed looping), console output quiet for 10min (assumed stalled), disk space exceeds apprx 13GB (too greedy) & build time exceeds 2hrs (too slow).

I have tried a variety of settings in travis.yml but my main issue is that Travis-ci does not appear to honour the make options.

compiler: gcc before_install: - test -n $CC && unset CC script: make V=0 ar71xx.usb cache: directories: - *-src - downloaded deploy: provider: releases api_key: secure: MB9lt........ksqYs= file: "/images" skip_cleanup: true on: repo: nworbnhoj/gargoyle `

The V=0 appears to be ignored and all targets appear to attempt to build. Any ideas

[stopspazzing]

Yes, sort of, I have a longer running version of it working in my github: https://github.com/stopspazzing/gargoyle/blob/master/.travis.yml

Still fails due to the verbose, I have even went as far as editing the make file to disable verbose but doesn't seem to do much and is extremely annoying. Making it threaded could speed up the build time also, its default set to auto

[nworbnhoj]

I am up to my armpits in the Makefile as we speak. I have messed about with a variety of settings including verbosity and threading etc as you suggest but it has become necessary to go a bit deeper.

One interesting thing that I am coming to understand is that the verbosity shown on the travis-ci screen at runtime is different to the verbosity captured in the raw log. Strangely when travis terminates the job due to a quiet 10 minutes (or 20 with travis_wait) is seems to be watching the screen output rather than the raw output.

Once the verbosity problem is nailed then there will be a disk space problem (deleting gargoyle/*-src/build_dir ASAP goes a long way to solving) and a time limit issue ...

I will report back when (if) I get somewhere ...

[stopspazzing]

Possible to make a test build which only checks if everything is there to compile correctly but only compiles a generic file?

[nworbnhoj]

As a type of dependency check? I do not understand the need?

[stopspazzing]

Yeah, guess you can say as a dependency check. Did you have an update to this?

[stopspazzing]

Farthest I have gotten. https://travis-ci.org/stopspazzing/Gargoyle-Firmware-Responsive

Anyway to use sdk to skip needing to compile toolkit? If I could cache this would be much faster but requires a special account. Is the website hosted on a VPS? If so can use Jenkins to do the builds and auto upload the binaries even.

[stopspazzing]

K so update to this cause this is very important project. Our best bet is to make a special script which is used for testing only which for example, breaks up the jobs into multiple jobs, why? Because 50min is the longest a job can run before travis kills it, so as long as we dont have a job last longer than 50min, can have as many jobs as needed.

So, if we could start a simple script broken into separate jobs:

1. Create all needed folders to finish the compile.
2. Compress all images,css,js files move them to compressed folders.
3. Use openwrt toolkit to compile packages faster if possible, since we don't need a custom toolkit, only packages, kernels, etc.
4. Only compiles a single image to verify the process works, example "default" image.
5. If compiles without error, continue compiling each chipset as seperate jobs.
6. Upload all binaries to github for distribution after successful compiling jobs.

OR... Setup a VPS with Jenkins and have it do it all using current scripts, then upload binaries.

[lantis1008]

My VPS often sits around doing not much.

[stopspazzing]

Well if thats the case, have you used jenkins? Ever set it up? If you want to get started it should be pretty easy and straight forward to install. Many plugins can be added also.

https://jenkins.io/index.html

There are ways of added a badge displaying if current build has passed or not. Honestly not sure why this project didnt have this setup already considering the complexities and modifications to openwrt default code that gargoyle adds.

[lantis1008]

@ispyisail have you got a rough number for the storage size required for a full build (all targets)? Not just the final images but all the built packages and source.

My VPS is currently limited in size

[stopspazzing]

Mine shows 11.8GB for just ar71xx.usb. How much space do you have?

[lantis1008]

50 total. About 5 being used personally.

[stopspazzing]

Oh, then just set each target to build with separate commands, and afterward each target build upload the files to github, then remove them then start next target, rise and repeat.

[ispyisail]

If I've done this correct

user@Linux:~/gargoyle$ du -hs ~/gargoyle/
55G     /home/user/gargoyle/
user@Linux:~/gargoyle$

55 Gig does seam to be on the large size?

[ericpaulbishop]

There's only about 15GB left on the main Gargoyle VPS, and I'd like to leave room for future builds, so if we could use a different VPS for this, that would be great (especially since I'm not sure if 15 GB will be enough)

ispyisail -- I don't think it will take 55GB per build. My guess is a lot of that is just the leftover stuff in the downloaded folder... but I could be wrong. If you wipe the folder do, a fresh git clone and do a build (note this will take longer, because you have to re-download everything), I suspect it will take less space.

[lantis1008]

I'll look into putting mine to work. Happy to help.

[stopspazzing]

Awesome, let me know if you need any help setting up Jenkins. Or adding badges from it to gargoyle read me so it shows whether recent build failed or was success, or how to upload binaries to github automatically, etc. This will fix the automated testing issue once set up. Still not sure how you guys went along without automated testing... :smile:

[stopspazzing]

Guessing this is abandoned? Want me to close this?

[lantis1008]

Up to you. I test compile before pushing most changes. Usually only on a single target.