ctrf-io / go-ctrf-json-reporter

A Go JSON test results reporter that follows the CTRF schema
https://ctrf.io
MIT License
5 stars 3 forks source link

Include filename and line number (closes #10) #11

Closed dmavrommatis closed 1 month ago

dmavrommatis commented 1 month ago

Addressing https://github.com/ctrf-io/go-ctrf-json-reporter/issues/10

The failed tests usually have the filename and the line number at the start of the printed message.

This PR uses a regex matcher to find that information and add it to the generated CTRF json.

cat ctrf-report.json 
{
  "results": {
    "tool": {
      "name": "gotest"
    },
    "summary": {
      "tests": 1,
      "passed": 0,
      "failed": 1,
      "pending": 0,
      "skipped": 0,
      "other": 0,
      "start": 1728408248844,
      "stop": 1728408248844
    },
    "tests": [
      {
        "name": "Test",
        "status": "failed",
        "duration": 0,
        "suite": "command-line-arguments",
        "message": "=== RUN   Test\n    whatever_test.go:156: \n        \tError Trace:\t/home/dmavrommatis/workspace/whatever_test.go:156\n        \tError:      \tWomp womp\n        \tTest:       \tTest\n--- FAIL: Test (0.00s)\n",
        "trace": "whatever_test.go:156",
        "filepath": "whatever_test.go:156"
      }
    ]
  }
}
dmavrommatis commented 1 month ago

I see the trace and path being set correctly but I still do not see the annotation being created correctly. which field is passed to the annotation? trace or filepath?

and does it do any glob find in the repository to find the file or not?

asherf commented 1 month ago

readme needs to be updated to indicate we populate this new field. https://github.com/ctrf-io/go-ctrf-json-reporter/blob/main/README.md#test-object-properties

coveralls commented 1 month ago

Pull Request Test Coverage Report for Build 11241118650

Details


Changes Missing Coverage Covered Lines Changed/Added Lines %
reporter/reporter.go 0 10 0.0%
<!-- Total: 0 10 0.0% -->
Files with Coverage Reduction New Missed Lines %
reporter/reporter.go 1 0.0%
<!-- Total: 1 -->
Totals Coverage Status
Change from base Build 10888553733: -1.1%
Covered Lines: 99
Relevant Lines: 274

💛 - Coveralls
dmavrommatis commented 1 month ago

will address the comments when I find some time

dmavrommatis commented 1 month ago

I tested this and the annotations created by the CTRF reporter afterwards are still not working correctly (maybe it will work with -fullpath but haven't tried).

Unfortunately I do not have much time to work on this so I will close the PR for now until someone else picks it up.