FR: add support for test results from autoconf (autotest)

[GitMensch]

autotest (provided by autoconf) is a very common framework for running tests in POSIX environments and often is responsible for the make check target. While it is possible to run the testsuite via MSYS/cygwin already it isn't possible to get the test results outside of console, at least from looking at the docs the only option are currently XML based test results while autotest produce log files (quite easy to read as a human and if checked for the blocks it should be possible to read them with the machine, too). Sample output:

## ------------------ ##
## Running the tests. ##
## ------------------ ##
testsuite: starting at: Sun Jul  3 22:08:55 CEST 2016
1. Compiler outputs (used_binaries.at:21): ok     (0m0.134s 0m0.246s)
2. Source file not found (used_binaries.at:46): ok     (0m0.004s 0m0.029s)
....
786. GLOBAL FD (2) (run_fundamental.at:1316): ok     (0m0.077s 0m0.168s)
testsuite: ending at: Sun Jul  3 22:09:44 CEST 2016
testsuite: test suite duration: 0h 0m 49s

## ------------- ##
## Test results. ##
## ------------- ##

ERROR: All 786 tests were run,
2 failed unexpectedly.

## ------------------------ ##
## Summary of the failures. ##
## ------------------------ ##
Failed tests:
GnuCOBOL 2.0 test suite: GnuCOBOL Tests test groups:

 NUM: FILE-NAME:LINE     TEST-GROUP-NAME
      KEYWORDS

 247: run_fundamental.at:1366 GLOBAL FD (3)
      fundamental
 248: run_fundamental.at:1416 GLOBAL FD (4)
      fundamental

Sample file (without any failures) testsuite.log.TXT

[GitMensch]

@IlyaFinkelshteyn As you've closed the discussion about adding test results for not-yet-supported formats: Where should I ask about this?

Additional there is another test runner for a second series of tests that creates a completely different format, which I could parse via script if I only knew how to specify a complete test result by appveyor command (without specifying test options "Framework" [in this case "script"] and "FileName" [would be multiple file names per test])...

In general: Does anyone see an option to get the autotest log file parsed officially?

[GitMensch]

ping @IlyaFinkelshteyn

[FeodorFitsner]

You can do a parser and then just upload test results via REST API or command line: https://www.appveyor.com/docs/build-worker-api/#add-tests

[GitMensch]

@FeodorFitsner The only current way to manually add tests via command line is still: manually convert your test results into a supported framework and have some exectuable you'd specify, then add the result via cmd line.

I've asked for an integrated option to have the standard formatted autotest results (the format existed long before any of the supported frameworks and still works perfectly fine) as I think many projects can benefit of this.

If this isn't possible it would be good to at least have an option to specify a test completely on the command line (without the need to specify any framework or executable).