ctrf-io / playwright-ctrf-json-reporter

A Playwright JSON test results reporter that follows the CTRF schema
https://ctrf.io
MIT License
31 stars 1 forks source link

Include all child steps for test steps #9

Open apatelia opened 3 hours ago

apatelia commented 3 hours ago

Currently, in CTRF report, all the test steps provide only "name" and "status" of given step.

Playwright provides many more details for each of the step, including a step's category, execution duration, location in source code and all the child steps.

Now that the 'step' object has been updated to include 'extra' property, child steps and other step details should be included in the CTRF report for Playwright.

For example, following test is currently reported without any steps.

test("has title @title", async ({ page }) => {
  await page.goto("https://playwright.dev/");

  await expect(page).toHaveTitle(/Playwright/);
});
{
  "name": "has title @title",
  "status": "passed",
  "duration": 1140,
  "start": 1729428643,
  "stop": 1729428644,
  "rawStatus": "passed",
  "tags": [
    "@title"
  ],
  "type": "e2e",
  "filePath": "/path-to-project-dir/tests/example.spec.ts",
  "retries": 0,
  "flaky": false,
  "steps": [],
  "suite": "chromium > example.spec.ts",
  "extra": {
    "annotations": []
  }
}

Ideally, it should be reported with both the steps included in steps property. For CTRF to be useful for debugging the tests, all step details including child steps, are needed to be on the report.

It should also be noticed that currently, none of the steps inside any hooks like beforeAll(), beforeEach(), afterAll() and afterEach() are reported in CTRF json.

apatelia commented 3 hours ago

I have opened a pull request for the same.

10