Travis-CI setup on modules.perl6.org

[zaucker]

Many modules are not setup for automated testing. As a minimal test coverage I would see that the module is at least installable with panda. I wonder if Travis-integration should not even be a mandatory requirement before a module can be published on modules.perl6.org

[zaucker]

One way to get there: create a .travis.yml file for every module you install (and which is not Travis-tested) and send a pull request to the author including the hint to activate automated testing of the module at https://travis-ci.org/USERNAME/MODULENAME

From my own experience: it is useful to fork the module first and test the Travis-testing before submitting the pull request (a failure might be a problem in the .travis.yml file).

[azawawi]

Great idea :+1:

Though I would not go to the mandatory level but I suggest issuing visible warnings of what is wrong with the module (e.g. audit log). Also we need to write clear instructions for the module writer to follow.

We can also add a warning sign near the module name in http://modules.perl.org 's table. When it is clicked, a modal dialog is shown that clearly states what needs to be fixed and how to do it.

[sylvarant]

Ocaml's opam repository has a set up where travis is used to check if the package manager can successfully install the new package proposed in a pull request.

As far as I can tell they have set up their travis.yml to run a script travis-ci.sh that through some sed trickery extracts all the needed info from the pull request and then attempts a build

[ugexe]

You can't make travis results mandatory when you can have modules that are expected to work on non-Linux operating systems but maybe not Linux itself.

[zaucker]

@ugexe: agreed. I take that suggestion back, then. There should probably be some mechanism to indicate that a module can only be used on certain OSes (perhaps one already exists?) and the Travis-column on the modules page should then show "n/a" (not applicable) instead of "not set up".

[ugexe]

perl5 (through metacpan) sort of does this except from CPAN testers matrix results. A perl6 version of this exists (although lacking enough testers to make it very useful) at http://testers.perl6.org, although maybe the possibility of cpan integration would involve integration with their test matrix as well.

For instance: http://testers.perl6.org/dist/A/Acme::Meow/%3Cunknown%3E.html (ignoring the obvious URI problem) shows you the results of various OS on different versions of rakudo

[autarch]

I'd note that AppVeyor is basically Travis for Windows. I've used it for a few perl5 modules and it works well. I don't know what it'd take to get perl6 testable via AppVeyor, but between Travis & AppVeyor it'd be possible to cover linux, OSX, and Windows.

This isn't quite a substitute for what CPANTesters does though, since that has the potential to test on way more variations in each platform in terms of underlying libraries, etc.

[ugexe]

@autarch I looked into AppVeyor; Here is a working example (configuration + badge): https://github.com/ugexe/zef/commit/b63ce3dcd8f6135747c814ee6464a7dcb7ac30d2

[preaction]

If the CPANTesters Metabase supports labelling a report with what language it is for, and p6 gets its own CPAN::Reporter module, and CPANTesters supports that language field to isolate the language's distributions and authors databases, then CPANTesters could be used for this.

Metabase should likely be straightforward if not easy. A CPAN::Reporter to write to Metabase, also straightforward. CPANTesters is not. I will be happy to assist any volunteers, but it will likely be weeks before I can deliver even a mockup or a spec for what changes must be made to CPANTesters.

I plan to make multiple languages part of the feature list of CPANTesters, but I can't give any sense of a timeframe on it. More discussion can happen in #cpantesters-discuss on irc.perl.org.