ctrf-io / go-ctrf-json-reporter

A Go JSON test results reporter that follows the CTRF schema
https://ctrf.io
MIT License
5 stars 3 forks source link

collect failed test error message #9

Closed asherf closed 2 months ago

asherf commented 2 months ago

fixes https://github.com/ctrf-io/go-ctrf-json-reporter/issues/1

in golang json test results this is how test failure looks like:

{"Time":"2024-09-13T23:49:11.52917925Z","Action":"pass","Package":"userclouds.com/console/internal/api","Test":"TestTenant/TestRenameTenant","Elapsed":0.18}
{"Time":"2024-09-13T23:49:11.529187455Z","Action":"output","Package":"userclouds.com/console/internal/api","Test":"TestTenant/TestSaveTenantConfigWithExistingPageParametersAndMessageElements","Output":"    /home/runner/work/userclouds/userclouds/console/internal/api/tenant_test.go:106: assertion failed\n"}
{"Time":"2024-09-13T23:49:11.529192474Z","Action":"output","Package":"userclouds.com/console/internal/api","Test":"TestTenant/TestSaveTenantConfigWithExistingPageParametersAndMessageElements","Output":"         got: {\"error\":\"an unspecified error occurred\",\"request_id\":\"adfb3363-a1ad-4c14-bf3c-577374267578\"}\n"}
{"Time":"2024-09-13T23:49:11.529198756Z","Action":"output","Package":"userclouds.com/console/internal/api","Test":"TestTenant/TestSaveTenantConfigWithExistingPageParametersAndMessageElements","Output":"              \n"}
{"Time":"2024-09-13T23:49:11.529203124Z","Action":"output","Package":"userclouds.com/console/internal/api","Test":"TestTenant/TestSaveTenantConfigWithExistingPageParametersAndMessageElements","Output":"              (wrapped) (File infra/jsonclient/client.go:420, in func2)\n"}
{"Time":"2024-09-13T23:49:11.529207342Z","Action":"output","Package":"userclouds.com/console/internal/api","Test":"TestTenant/TestSaveTenantConfigWithExistingPageParametersAndMessageElements","Output":"              (wrapped) (File infra/jsonclient/client.go:284, in func1)\n"}
{"Time":"2024-09-13T23:49:11.52921163Z","Action":"output","Package":"userclouds.com/console/internal/api","Test":"TestTenant/TestSaveTenantConfigWithExistingPageParametersAndMessageElements","Output":"              (wrapped) (File infra/jsonclient/client.go:253, in Put)\n"}
{"Time":"2024-09-13T23:49:11.529215908Z","Action":"output","Package":"userclouds.com/console/internal/api","Test":"TestTenant/TestSaveTenantConfigWithExistingPageParametersAndMessageElements","Output":"        want: \u003cnil\u003e\n"}
{"Time":"2024-09-13T23:49:11.529220186Z","Action":"output","Package":"userclouds.com/console/internal/api","Test":"TestTenant/TestSaveTenantConfigWithExistingPageParametersAndMessageElements","Output":"        \n"}
{"Time":"2024-09-13T23:49:11.529228933Z","Action":"output","Package":"userclouds.com/console/internal/api","Test":"TestTenant/TestSaveTenantConfigWithExistingPageParametersAndMessageElements","Output":"--- FAIL: TestTenant/TestSaveTenantConfigWithExistingPageParametersAndMessageElements (0.42s)\n"}
{"Time":"2024-09-13T23:49:11.534088531Z","Action":"output","Package":"userclouds.com/console/internal/api","Test":"TestTenant/TestSaveTenantConfigWithExistingPageParametersAndMessageElements","Output":"2024/09/13 23:49:11 ac3567c2-6d36-46cb-8746-cde5329ef787: HTTP     200 | 42.226839ms (6689B -\u003e 3355B) | DB 1 3.935966ms | Cache: c:9 h:9 m:0 1.336369ms | console.test.userclouds.tools:39997 | 00000000-0000-0000-0000-000000000000 | POST /api/tenants/bf271a1e-2920-4893-b63c-ae163625040e/plexconfig\n"}
{"Time":"2024-09-13T23:49:11.536304459Z","Action":"output","Package":"userclouds.com/console/internal/api","Test":"TestTenant/TestSaveTenantConfigWithExistingPageParametersAndMessageElements","Output":"2024/09/13 23:49:11 [D] jsonclient request POST /api/tenants/bf271a1e-2920-4893-b63c-ae163625040e/plexconfig took 52.701145ms request id: ac3567c2-6d36-46cb-8746-cde5329ef787\n"}
{"Time":"2024-09-13T23:49:11.536344514Z","Action":"fail","Package":"userclouds.com/console/internal/api","Test":"TestTenant/TestSaveTenantConfigWithExistingPageParametersAndMessageElements","Elapsed":0.42}

tested locally:

➜  go-ctrf-json-reporter git:(extract-err) go build -o go-ctrf-json-reporter   cmd/go-ctrf-json-reporter/main.go
➜  go-ctrf-json-reporter git:(extract-err) ✗ cat ~/Downloads/go-test-json-results.json | ./go-ctrf-json-reporter
go-ctrf-json-reporter: successfully written ctrf json to ctrf-report.json
➜  go-ctrf-json-reporter git:(extract-err) ✗ cat ctrf-report.json | jq -r '.results.tests[] | select(.status == "failed") | .message'
    /home/runner/work/userclouds/userclouds/console/internal/api/tenant_test.go:106: assertion failed
         got: {"error":"an unspecified error occurred","request_id":"adfb3363-a1ad-4c14-bf3c-577374267578"}

              (wrapped) (File infra/jsonclient/client.go:420, in func2)
              (wrapped) (File infra/jsonclient/client.go:284, in func1)
              (wrapped) (File infra/jsonclient/client.go:253, in Put)
        want: <nil>

--- FAIL: TestTenant/TestSaveTenantConfigWithExistingPageParametersAndMessageElements (0.42s)
2024/09/13 23:49:11 ac3567c2-6d36-46cb-8746-cde5329ef787: HTTP     200 | 42.226839ms (6689B -> 3355B) | DB 1 3.935966ms | Cache: c:9 h:9 m:0 1.336369ms | console.test.userclouds.tools:39997 | 00000000-0000-0000-0000-000000000000 | POST /api/tenants/bf271a1e-2920-4893-b63c-ae163625040e/plexconfig
2024/09/13 23:49:11 [D] jsonclient request POST /api/tenants/bf271a1e-2920-4893-b63c-ae163625040e/plexconfig took 52.701145ms request id: ac3567c2-6d36-46cb-8746-cde5329ef787

--- FAIL: TestTenant (11.24s)
coveralls commented 2 months ago

Pull Request Test Coverage Report for Build 10858212570

Details


Changes Missing Coverage Covered Lines Changed/Added Lines %
reporter/reporter.go 0 18 0.0%
<!-- Total: 0 18 0.0% -->
Totals Coverage Status
Change from base Build 10818405964: -2.4%
Covered Lines: 99
Relevant Lines: 266

💛 - Coveralls