search

alexneo2003 / playwright-azure-reporter

Playwright Azure DevOps Reporter
MIT License
67 stars 11 forks source link
automated-testing azure-devops playwright reporter reporting testing typescript

readme

Playwright Azure Reporter

GitHub npm (scoped) npm npm

A must read!

Since version 1.5.0 reporter allows using configurationIds to publish results for different configurations e.g. different browsers Necessarily defining testRun.configurationIds or/and testPointMapper function in reporter config, otherwise reporter will be publishing results for all configurations

Since version 1.9.0 reporter allows you to use test tags as Playwright it implemented in version 1.42.0 You can define test cases ids in new format, but you still can use old format with test case id in test name

Example:

test.describe('Test suite', () => {
  test('Test name @tag1 @tag2', {
    tag: ['@[1]'] // <<-- test case id
  } () => {
    expect(true).toBe(true);
  });
});

but you should define your Azure DevOps test case id in format @[1] where 1 is your test case id in square brackets and @ is required prefix for playwright to recognize tags

How to integrate

Install package

npm install @alex_neo/playwright-azure-reporter

or

yarn add @alex_neo/playwright-azure-reporter

Usage

You must register an ID already existing test cases from Azure DevOps before running tests.

You need write testCaseId wraped in square brackets at the test name.

You can define multiple test cases for a single test with next format:

For example:

describe('Test suite', () => {
  test('[1] First Test', () => {
    expect(true).toBe(true);
  });

  test('Correct test [3]', () => {
    expect(true).toBe(true);
  });

  test.skip('Skipped test [4]', () => {
    expect(true).toBe(true);
  });

  test('[6] Failed test', () => {
    expect(true).toBe(false);
  });

  test('[7] Test seven [8] Test eight [9] Test nine', () => {
    expect(true).toBe(true);
  });

  test('[10,11,12] Test ten, eleven, twelve', () => {
    expect(true).toBe(true);
  });

  test('[13, 14, 15] Test thirteen, fourteen, fifteen', () => {
    expect(true).toBe(true);
  });

  test('[16, 17, 18] Test sixteen, seventeen, eighteen [19] Test nineteen', () => {
    expect(true).toBe(true);
  });
});

Or you can use tags to define test cases ids (since v1.9.0) (read more here):

test.describe('Test suite', () => {
  test('Test name', {
    tag: ['@[1]', '@smoke', '@slow']
  } () => {
    expect(true).toBe(true);
  });
});

Configure Playwright Azure Reporter with playwright-azure-reporter package.

playwright.config.ts

import { PlaywrightTestConfig } from '@playwright/test';
import { AzureReporterOptions } from '@alex_neo/playwright-azure-reporter/dist/playwright-azure-reporter';

/**
 * See https://playwright.dev/docs/test-configuration.
 */
const config: PlaywrightTestConfig = {
  testDir: './tests',
  timeout: 30 * 1000,
  expect: {
    timeout: 5000,
  },
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 1 : undefined,
  reporter: [
    ['list'],
    [
      '@alex_neo/playwright-azure-reporter',
      {
        orgUrl: 'https://dev.azure.com/your-organization-name',
        token: process.env.AZURE_TOKEN,
        planId: 44,
        projectName: 'SampleSample',
        environment: 'AQA',
        logging: true,
        testRunTitle: 'Playwright Test Run',
        publishTestResultsMode: 'testRun',
        uploadAttachments: true,
        attachmentsType: ['screenshot', 'video', 'trace'],
        testCaseIdMatcher: /@\[(\d+)\]/,
        testRunConfig: {
          owner: {
            displayName: 'Alex Neo',
          },
          comment: 'Playwright Test Run',
          // the configuration ids of this test run, use
          // https://dev.azure.com/{organization}/{project}/_apis/test/configurations to get the ids of  your project.
          // if multiple configuration ids are used in one run a testPointMapper should be used to pick the correct one,
          // otherwise the results are pushed to all.
          configurationIds: [1],
        },
      } as AzureReporterOptions,
    ],
  ],
  use: {
    screenshot: 'only-on-failure',
    actionTimeout: 0,
    trace: 'on-first-retry',
  },
};

export default config;

Configuration

Reporter options (* - required):

  import { TestCase } from '@playwright/test/reporter'
  import { TestPoint } from 'azure-devops-node-api/interfaces/TestInterfaces'

  testPointMapper: async (testCase: TestCase, testPoints: TestPoint[]) => {
    switch(testCase.parent.project()?.use.browserName) {
      case 'chromium':
        return testPoints.filter((testPoint) => testPoint.configuration.id === '3');
      case 'firefox':
        return testPoints.filter((testPoint) => testPoint.configuration.id === '4');
      case 'webkit':
        return testPoints.filter((testPoint) => testPoint.configuration.id === '5');
      default:
        throw new Error("invalid test configuration!");
    }
  }

Usefulness