make clearer a 'make test' error

[hectoralos]

I get:

$ make test
apertium-regtest test
Corpus 1 of 1: analysis
  7/9 (77.78%) tests pass (0/7 (0.0%) match gold)

There were changes! Rerun in interactive mode to update tests.
Changed corpora: analysis
make: *** [Makefile:928: test] Error 1

It would be easier if the message explained what has to be done to "rerun in interactive mode". Moreover, a search in the wiki of "rerun in interactive mode" or "run in interactive mode" does not give meaningful results.

[TinoDidriksen]

While it could be more informative, I'd say it's natural to run apertium-regtest --help to see available options.

In this case, you'd want to run apertium-regtest -c analysis cli or apertium-regtest -c analysis web

[hectoralos]

Even the name appears in the first line of the message, I'm not sure that everybody may understand that this a programme that can be called.

[unhammer]

I agree this could be clearer – other people have commented that they don't understand what to do when this fails. The above PR changes it to say Run 'apertium-regtest cli' to update tests which might help a little bit, but perhaps it could be even clearer?

It's especially important to be clear and helpful because the change from wiki-tests to regtest was not done by the people actually running and fixing the tests

[mr-martian]

2021387 implements fuller instructions by default and a quiet in case anyone is running this by hand and doesn't want that much output.

⏚ daniel@computer-name-here:~/apertium/apertium-hbo master(+1/-1)+* 3s ± ../apertium-regtest/apertium-regtest.py test
Corpus 1 of 4: analysis
  1/1 (100.0%) tests pass (0/1 (0.0%) match gold)

Corpus 2 of 4: generation
  0/1 (0.0%) tests pass

Corpus 3 of 4: קטל-paradigm
  147/147 (100.0%) tests pass (140/147 (95.24%) match gold)

Corpus 4 of 4: אמר-paradigm
  147/147 (100.0%) tests pass (18/147 (12.24%) match gold)

There were changes!
The tests need to be updated.
The copora that contain changes are: generation
This can be done in a browser by running

  apertium-regtest web

or on the command line with

  apertium-regtest cli

A specific corpus can be edited with `-c`, for example

  apertium-regtest -c generation web

⏚ daniel@computer-name-here:~/apertium/apertium-hbo master(+1/-1)+* 1 ± ../apertium-regtest/apertium-regtest.py -q test
Corpus 1 of 4: analysis
  1/1 (100.0%) tests pass (0/1 (0.0%) match gold)

Corpus 2 of 4: generation
  0/1 (0.0%) tests pass

Corpus 3 of 4: קטל-paradigm
  147/147 (100.0%) tests pass (140/147 (95.24%) match gold)

Corpus 4 of 4: אמר-paradigm
  147/147 (100.0%) tests pass (18/147 (12.24%) match gold)

There were changes! Run `apertium-regtest cli` to update tests.
Changed corpora: generation

Does that look good?

[hectoralos]

It seems a good and clear enough addition for me. Note the typo in "copora".

[TinoDidriksen]

Can you detect a TTY and change whether -q is default based on that? Don't want the long output in build logs, and they are !isatty

[TinoDidriksen]

I just realized that when run under make then stdout is not a TTY either, so meh.

[mr-martian]

I fixed the typo and set it to also give the shorter output on AP_REGTEST_QUIET=yes.