elaichenkov / playwright-zephyr

Zephyr reporter for the Playwright
MIT License
29 stars 17 forks source link

Reporting comments for error and stack trace variables are appearing uninteligable in Zephyr Scale cloud #45

Open KeithHEnnis opened 10 months ago

KeithHEnnis commented 10 months ago

I had a simple set of Typescript Playwright API tests set up and reporting just fine (error and stack trace messages reflected playwright's test results ) with the provided zephyr cloud reporter settings,

I progressed my test design by creating helper object classes and changing the .spect.ts import of test from @playwright/test to my fixture.ts which extends test via import { test as base } from '@playwright/test'; and substantiate the class making use of it's constructor

export const test = base.extend<{ 
  importedHelperClass: ImportedHelperClass;
}>({
    importedHelperClass:async ({request}, use) => {
        await use(new ImportedHelperClass(request));      
    }
});

Since then the error and stack trace messages recorded in Zephyr Scale are far from human readable:

❌ Error Message: expect(received).toBe(expected) // Object.is equality

Expected: 400 Received: 200

🧱 Stack Trace: Error: expect(received).toBe(expected) // Object.is equality

Expected: 400 Received: 200

Is there something to further configure in the reporter? or is this an issue with the repo not handling playwright test being extended?

elaichenkov commented 10 months ago

Hey, Thank you for reporting the issue. However, that looks odd because the reporter uses the playwright's native error message (e.g. result.error.message). Anyway, I'll try to investigate it more.

KeithHEnnis commented 10 months ago

I thought it odd too, especially since the playwright extension for VSCode I use reports just fine per test, and running the whole thing locally or in docker switched back to the html reporter shows no signs of corrupted error output. Thanks for investigating.