Implement moodle-plugin-ci start/finish command

[kabalin]

This will be useful to achieve a series of tests running behaviour similar to Travis, while running in different environment (e.g. GitHub Actions or GitLab). We got something similar already for testing Workplace on GitLab but as a separate wrapping script. Normally test will stop on encountering error (but not in Travis), so we may need a wrapper that will collect exit statuses of each moodle-plugin-ci test execution and then report result at the end, e.g.

moodle-plugin-ci start # This will indicate that each command should exit with 0 but real status will be collected in env var.
moodle-plugin-ci phplint
moodle-plugin-ci phpcpd
moodle-plugin-ci phpmd
moodle-plugin-ci codechecker
....
moodle-plugin-ci finish # Check what we collected, list tests with non-zero status, exit with error code if needed.

The last command will produce depending on result: All tests were successful or for example Failed tests: phplint, codechecker and exit with correct status so environment will report this as failed or succeeded.

[marinaglancy]

Just commenting here that is a way to do it in github actions, for example:

  - name: Moodle PHPDoc Checker
    if: ${{ always() }}
    run: moodle-plugin-ci phpdoc

  - name: Validating
    if: ${{ always() }}
    run: moodle-plugin-ci validate

It is still however a problem with gitlab ci :(

[mark-webster-catalyst]

I'm working on getting this running in Gitlab at the moment and have a workaround for this issue. You can redirect the exit code to a variable like this

script:
  - exit_code=0
  - moodle-plugin-ci phplint || exit_code+="$?"
  - moodle-plugin-ci validate || exit_code+="$?"
  - >
    if [ $exit_code -ne 0 ]; then
      echo "One or more jobs failed."
      exit 0
    fi

You can be more granular with the variables of you want to know exactly which jobs failed, but like this it will at least allow the stage to pass if all the checks completed, regardless of whether or not there are errors. So your pipeline will only "fail" if something actually goes wrong and the tests don't run properly.