more configs

[tpimh]

(not a duplicate of #21)

I think additional configs may be tested on travis. defconfig is good, but it also makes sense testing tinyconfig (+ enable elf support and tty driver, it will run init and print boot log to console), allyesconfig and allmodconfig. And run all in parallel with travis. Will it be good for testing or just a waste of resources?

[arndb]

allmodconfig and allyesconfig are generally intended for build testing, not runtime testing. Getting allyesconfig to boot would be a lot of work, though I have managed to boot an x86 allmodconfig (after disabling CONFIG_MAXSMP and CONFIG_CMDLINE_OVERRIDE), and Anders Roxell has managed to boot arm64 allmodconfig after overriding around a dozen different options. This was all using gcc, so I'd expect additional problems with clang. Getting those to work in clang would be great of course, but I'm not sure about running them continuously unless we first make it reliable on gcc.

[nathanchance]

Not to mention that Travis is our biggest bottleneck. We can't even do a full defconfig build in the allotted time (50 minutes), we're just building the kernel image rather than all of the modules as well.

[nickdesaulniers]

I can ask my boss if we can pay for beefier travis machines, but I think kernel-ci may be better for heavier lifting.

I wonder if we can have more than one cron jobs? For example, I wonder if we could do daily builds and per pull request builds of a subset of targets, then a weekly build of all targets?

[nathanchance]

Yeah I am sure Travis would let us break down the matrix into different subsets that we can target with different actions.

[tpimh]

I contacted Travis support and they said that they can increase time limit to 180 minutes by request, but no more than that.

[nickdesaulniers]

@tpimh neat, good to know. Thanks for doing that. I wonder if they have a limit on the size of our testing "matrix." Because I see that as being the limiting factor for the number of configs we can test in a given run.

[tapaswenipathak]

Hello, reading the repository I think there is only common and tt config right now. What is this ticket's status on allyesconfig and allmodeconfig? This seems interesting, let me know if any of you is already working on it.

[nickdesaulniers]

I expect an allyesconfig to timeout on travisCI (thus making that config untestable). TravisCI gives us 50 minutes per test before it considers a build failed/timed out.

[nathanchance]

Yeah, we already cut it close on a defconfig with no modules built on a completely cold cache (41 minutes on average).

[tapaswenipathak]

Huh I see, should be fine, do you test it after increasing limits on your own Travis accounts or you have a Travis account for this org?

[tpimh]

I can request to extend the build time (it's free) once it hits the limit. I already did it for another project.

[nickdesaulniers]

I can request to extend the build time (it's free) once it hits the limit. I already did it for another project.

Neat. Do how did you go about doing that? What did they extend it to? Which project?

[tpimh]

I was trying to build a toolchain on travis and ended up getting extra time. The maximum is 180 minute, I'm sure it's not possible to get more than 3 hours per job, but it should be enough. It's done by contacting support after a job fails because it hits the time limit.

[tapaswenipathak]

@tpimh will do if required, leave it on me.