Feature Request(s): Use Playwright's TestCase.id or allow report dump to file

[jeremy-daley-kr]

I might be missing something, but it seems the testCaseId in this reporter's terminology differs from the actual Playwright type that's called TestCase.id?

The data that gets sent to ReportPortal gets stored, seemingly without any correlating Item field to what Playwright sends to its own reporters (json reporter for instance)... So instead I'm left with creating a concatenation used the same way this reporter generates codeRef, which can be very big, and not friendly to passing/storing.

Is there a particular reason the Playwright TestCase.id wan't used instead?

Additionally, a different feature of dumping the report's data would help create that correlation, post-run.

All of this is in hopes of creating a tighter integration with our CI in Github. Ideally, a user would have the ability to click on a failed test's output link, and be directed to ReportPortal where a full Playwright integration would show its trace viewer.

[AmsterGet]

Hi @jeremy-daley-kr ! You're right. Right now we are not using TestCase.id from Playwright. It was added in Playwright 1.25, so we were initially forced to use our own id's. We have a backlog task to implement using Playwright's TestCase.id as RP testCaseId, so I assume this will be available by toggling some option in the config for backwards compatibility. So for this case, if I understand correctly, that would be enough for you to provide the necessary binding.

As for your second question, could you describe it a bit more in detail? Do you expect to see a link in CI for each specific failed test case in RP? Right now, all reporters print a link to the RP launch at the end of the entire run.

[jeremy-daley-kr]

As I started digging through the built-in JSON reporter more, I think I'm seeing that TestCase.id doesn't persist through retries, and perhaps that's one reason the testPath() concatenation is necessary for this reporter to keep retries grouped to the same actual test... but maybe that's the codeRef field's purpose?

Additionally, I didn't realize there's a link to the RP launch at the end of the run. Does that require a reporter config value be set? Perhaps we need to upgrade our current version (5.1.1).

As for your second question, could you describe it a bit more in detail?

Sure... simply put, writing a report-portal-results.json file to the results folder. I imagine this is the data I would see logged if debug: true. That's mostly the TLDR of that question. I don't need specific CI platforms (Github) integration, just the data to make correlation in my own integrations that then read the data via the RP API.

The final sentence was probably more of a "pie in the sky" RP platform feature request that would allow an integration here:

...It would somehow allow a more seamless end-to-end from failure to inspection, like I'm doing in my own integration:

(Github)

(self-hosted Playwright trace viewer)