mheap / phpunit-matcher-action

Add annotations to your PHPUnit tests when running under Github Actions
32 stars 6 forks source link

Add ability to customise the root folder #11

Closed mheap closed 2 years ago

mheap commented 2 years ago

I run tests in a docker container built in previous job so teamcity log output has container path instead of github workspace path expected by this action.

I would like to have an option to specify custom base path for the matcher.

@Xerkus Would this be enough to solve your issue?

Resolves #9

Xerkus commented 2 years ago

Yep, it works.

Interesting, however, that it does not produce annotation on PR diff page.

Matcher:

{"problemMatcher":[{"owner":"phpunit-failure","severity":"error","pattern":[{"regexp":"##teamcity\\[testFailed.+message='(.+)'.+details='\\s+/app/([^:]+):(\\d+)[^']+'","message":1,"file":2,"line":3}]}]}

Matched output line:

##teamcity[testFailed name='testResponse' message='Failed asserting that an array has the key |'non-existent|'.' details=' /app/test/Unit/RequestHandler/PingHandlerTest.php:30|n ' duration='6' flowId='1']

Annotations in action run summary:

Unit tests
Failed asserting that an array has the key |'non-existent|'.

Netlify matcher tester:

[
    {
        "message": "Failed asserting that an array has the key |'non-existent|'.",
        "file": "test/Unit/RequestHandler/PingHandlerTest.php",
        "line": "30",
        "severity": "error"
    }
]

I guess it is a limitation of if: failure()

      -
        name: "Unit tests with PhpUnit"
        id: phpunit
        run: |
          docker run --rm -e XDEBUG_MODE --pull never ${{ steps.test_image.outputs.image }} \
            vendor/bin/phpunit --testsuite unit --coverage-text
      -
        name: Configure matchers
        if: failure() && steps.phpunit.outcome == 'failure'
        uses: mheap/phpunit-matcher-action@add-custom-root-folder
        with:
          base_path: /app
      -
        name: "Unit tests with PhpUnit rerun for PR annotations"
        if: failure() && steps.phpunit.outcome == 'failure'
        run: |
          docker run --rm -e XDEBUG_MODE --pull never ${{ steps.test_image.outputs.image }} \
            vendor/bin/phpunit --teamcity --testsuite unit
Xerkus commented 2 years ago

Looks like repo content MUST be checkout for the annotations to be created, even if it is otherwise unused:

##[debug]Dropping file value '/home/runner/work/reponame/reponame/test/Unit/RequestHandler/PingHandlerTest.php'. Path does not exist

https://github.com/actions/runner/blob/6b4a95cdb1f5bca59c619dfda912e3d9c874d8b1/src/Runner.Worker/Handlers/OutputManager.cs#L299-L320

mheap commented 2 years ago

Excellent debugging skills! If you run actions/checkout before running with the annotations does it work?

Xerkus commented 2 years ago

Yes, it works like a charm.

First run with human readable output with text coverage report and if there are failures then tests are rerun with teamcity reporting format for annotations. Checkout step was crucial for inline diff annotations.

      -
        name: "Unit tests with PhpUnit"
        id: phpunit
        run: |
          docker run --rm -e XDEBUG_MODE --pull never ${{ steps.test_image.outputs.image }} \
            vendor/bin/phpunit --testsuite unit --coverage-text
      -
        name: Checkout code for annotations
        if: failure() && steps.phpunit.outcome == 'failure'
        uses: actions/checkout@v2
      -
        name: Configure matchers
        if: failure() && steps.phpunit.outcome == 'failure'
        uses: mheap/phpunit-matcher-action@add-custom-root-folder
        with:
          base_path: /app
      -
        name: "Unit tests with PhpUnit rerun for PR annotations"
        if: failure() && steps.phpunit.outcome == 'failure'
        run: |
          docker run --rm -e XDEBUG_MODE --pull never ${{ steps.test_image.outputs.image }} \
            vendor/bin/phpunit --teamcity --testsuite unit
mheap commented 2 years ago

Thanks for your help testing and debugging. That's now available on master 🙌