Spectral junit report has wrong paths to OpenAPI spec file

[chilcano]

Hello, I'm using Spectral CLI from Github Actions to lint OpenAPI spec files. Once triggered, it generates reports in different formats, even JUnit, which I do use from EnricoMi/publish-unit-test-result-action to publish it as Annotations in Github Action Summary.

The EnricoMi/publish-unit-test-result-action publishes successfully the report as annotations in the Summary view and in the Check runs view. However, the paths and links are wrong and if I click on any of them, it is redirected to Github Code showing the diff view.

I would like to publish the Spectral junit report as annotations with the right paths and links where after clicking on these links i can view the current OpenAPI spec file showing the error or warning in the right line. I've tried test_file_prefix without success.

• Github workflow I'm using: https://github.com/chilcano/how-tos/blob/main/.github/workflows/spectral-owasp-apisec.yaml
• Spectral junit report generated by the Github workflow (it can be downloaded from here: https://github.com/chilcano/how-tos/actions/runs/8602430956)

  <?xml version="1.0" encoding="utf-8"?>
  <testsuites>
  <testsuite package="org.spectral" time="0" tests="15" errors="0" failures="15" name="/home/runner/work/how-tos/how-tos/src/api_specs/petstore.yaml">
  <testcase time="0" name="org.spectral.owasp:api9:2023-inventory-access(#/servers/0)" classname="/home/runner/work/how-tos/how-tos/src/api_specs/petstore"><failure message="Declare intended audience of every server by defining servers[0].x-internal as true/false."><![CDATA[line 8, col 5, Declare intended audience of every server by defining servers[0].x-internal as true/false. (owasp:api9:2023-inventory-access) at path #/servers/0]]></failure></testcase>
  <testcase time="0" name="org.spectral.owasp:api8:2023-no-server-http(#/servers/0/url)" classname="/home/runner/work/how-tos/how-tos/src/api_specs/petstore"><failure message="Server URLs must not use http://. Use https:// or wss:// instead."><![CDATA[line 8, col 10, Server URLs must not use http://. Use https:// or wss:// instead. (owasp:api8:2023-no-server-http) at path #/servers/0/url]]></failure></testcase>
  <testcase time="0" name="org.spectral.owasp:api4:2023-integer-limit-legacy(#/paths/~1pets/get/parameters/0/schema)" classname="/home/runner/work/how-tos/how-tos/src/api_specs/petstore"><failure message="Schema of type integer must specify minimum and maximum."><![CDATA[line 21, col 18, Schema of type integer must specify minimum and maximum. (owasp:api4:2023-integer-limit-legacy) at path #/paths/~1pets/get/parameters/0/schema]]></failure></testcase>
  <testcase time="0" name="org.spectral.owasp:api4:2023-rate-limit(#/paths/~1pets/get/responses/200/headers)" classname="/home/runner/work/how-tos/how-tos/src/api_specs/petstore"><failure message="All 2XX and 4XX responses should define rate limiting headers."><![CDATA[line 28, col 19, All 2XX and 4XX responses should define rate limiting headers. (owasp:api4:2023-rate-limit) at path #/paths/~1pets/get/responses/200/headers]]></failure></testcase>
  <testcase time="0" name="org.spectral.owasp:api8:2023-define-cors-origin(#/paths/~1pets/get/responses/200/headers)" classname="/home/runner/work/how-tos/how-tos/src/api_specs/petstore"><failure message="Header `headers.Access-Control-Allow-Origin` should be defined on all responses."><![CDATA[line 28, col 19, Header `headers.Access-Control-Allow-Origin` should be defined on all responses. (owasp:api8:2023-define-cors-origin) at path #/paths/~1pets/get/responses/200/headers]]></failure></testcase>
  <testcase time="0" name="org.spectral.owasp:api4:2023-string-limit(#/paths/~1pets/get/responses/200/headers/x-next/schema)" classname="/home/runner/work/how-tos/how-tos/src/api_specs/petstore"><failure message="Schema of type string must specify maxLength, enum, or const."><![CDATA[line 31, col 22, Schema of type string must specify maxLength, enum, or const. (owasp:api4:2023-string-limit) at path #/paths/~1pets/get/responses/200/headers/x-next/schema]]></failure></testcase>
  <testcase time="0" name="org.spectral.owasp:api2:2023-write-restricted(#/paths/~1pets/post)" classname="/home/runner/work/how-tos/how-tos/src/api_specs/petstore"><failure message="This write operation is not protected by any security scheme."><![CDATA[line 43, col 10, This write operation is not protected by any security scheme. (owasp:api2:2023-write-restricted) at path #/paths/~1pets/post]]></failure></testcase>
  <testcase time="0" name="org.spectral.owasp:api4:2023-rate-limit(#/paths/~1pets/post/responses/201)" classname="/home/runner/work/how-tos/how-tos/src/api_specs/petstore"><failure message="All 2XX and 4XX responses should define rate limiting headers."><![CDATA[line 55, col 15, All 2XX and 4XX responses should define rate limiting headers. (owasp:api4:2023-rate-limit) at path #/paths/~1pets/post/responses/201]]></failure></testcase>
  <testcase time="0" name="org.spectral.owasp:api4:2023-string-limit(#/paths/~1pets~1%7BpetId%7D/get/parameters/0/schema)" classname="/home/runner/work/how-tos/how-tos/src/api_specs/petstore"><failure message="Schema of type string must specify maxLength, enum, or const."><![CDATA[line 74, col 18, Schema of type string must specify maxLength, enum, or const. (owasp:api4:2023-string-limit) at path #/paths/~1pets~1%7BpetId%7D/get/parameters/0/schema]]></failure></testcase>
  <testcase time="0" name="org.spectral.owasp:api4:2023-rate-limit(#/paths/~1pets~1%7BpetId%7D/get/responses/200)" classname="/home/runner/work/how-tos/how-tos/src/api_specs/petstore"><failure message="All 2XX and 4XX responses should define rate limiting headers."><![CDATA[line 77, col 15, All 2XX and 4XX responses should define rate limiting headers. (owasp:api4:2023-rate-limit) at path #/paths/~1pets~1%7BpetId%7D/get/responses/200]]></failure></testcase>
  <testcase time="0" name="org.spectral.owasp:api4:2023-integer-limit-legacy(#/components/schemas/Pet/properties/id)" classname="/home/runner/work/how-tos/how-tos/src/api_specs/petstore"><failure message="Schema of type integer must specify minimum and maximum."><![CDATA[line 97, col 12, Schema of type integer must specify minimum and maximum. (owasp:api4:2023-integer-limit-legacy) at path #/components/schemas/Pet/properties/id]]></failure></testcase>
  <testcase time="0" name="org.spectral.owasp:api4:2023-string-limit(#/components/schemas/Pet/properties/name)" classname="/home/runner/work/how-tos/how-tos/src/api_specs/petstore"><failure message="Schema of type string must specify maxLength, enum, or const."><![CDATA[line 100, col 14, Schema of type string must specify maxLength, enum, or const. (owasp:api4:2023-string-limit) at path #/components/schemas/Pet/properties/name]]></failure></testcase>
  <testcase time="0" name="org.spectral.owasp:api4:2023-string-limit(#/components/schemas/Pet/properties/tag)" classname="/home/runner/work/how-tos/how-tos/src/api_specs/petstore"><failure message="Schema of type string must specify maxLength, enum, or const."><![CDATA[line 102, col 13, Schema of type string must specify maxLength, enum, or const. (owasp:api4:2023-string-limit) at path #/components/schemas/Pet/properties/tag]]></failure></testcase>
  <testcase time="0" name="org.spectral.owasp:api4:2023-integer-limit-legacy(#/components/schemas/Error/properties/code)" classname="/home/runner/work/how-tos/how-tos/src/api_specs/petstore"><failure message="Schema of type integer must specify minimum and maximum."><![CDATA[line 115, col 14, Schema of type integer must specify minimum and maximum. (owasp:api4:2023-integer-limit-legacy) at path #/components/schemas/Error/properties/code]]></failure></testcase>
  <testcase time="0" name="org.spectral.owasp:api4:2023-string-limit(#/components/schemas/Error/properties/message)" classname="/home/runner/work/how-tos/how-tos/src/api_specs/petstore"><failure message="Schema of type string must specify maxLength, enum, or const."><![CDATA[line 118, col 17, Schema of type string must specify maxLength, enum, or const. (owasp:api4:2023-string-limit) at path #/components/schemas/Error/properties/message]]></failure></testcase>
  </testsuite>
  </testsuites>

  I think that Spectral CLI is not generating the junit xml file is a format that EnricoMi/publish-unit-test-result-action does expect. If so, I could edit the generated junit file in run-time.

In this case, what is the expected junit format what should be used to show the OpenAPI spec file in Github Code view with the specific annotation for the corresponding line like EnricoMi/publish-unit-test-result-action shows in its documentation (https://github.com/EnricoMi/publish-unit-test-result-action?tab=readme-ov-file#commit-and-pull-request-annotations)

I appreciate your ideas and support. Kind regards.

[EnricoMi]

I cannot see a file name in your example XML, which is why the links do not work as expected. The <testcase> tags need a file attribute that points to the file that cause the test failure. Ideally, you also have the line attribute. I think those information are currently contained in the <failure> CDATA content. That has to be used to populate file and line.

Here is an example: https://github.com/EnricoMi/publish-unit-test-result-action/blob/master/python/test/files/junit-xml/testsuite-root.xml

[EnricoMi]

Spectral junit report might benefit from this feature..

[chilcano]

Thanks @EnricoMi Let me generate a junit file with the expected format. I'll come back to you with my findings.

Regards.