Store PlayRight XRay Results

sn-sstone commented 1 month ago

Is your feature request related to a problem? Please describe. It would be nice to be able to store the results of the XRAY Reporter somewhere, either in a file or env variable.

What is "Results": Things like what is provided at the end of the test run, such as.

😀 Successfully sending test results to Jira

⏺  Browsers:          Setup, Webkit
⏺  Test plan:         SNP-3674
⏺  Test Duration:     3h 32m 
⏺  Tests ran:         113 (including reruns)
⏺  Tests passed:      0
⏺  Tests failed:      113
⏺  Flaky tests:       0
⏺  Skipped tests:     0

-------------------------------------

⏺  Test execution SNP-4579 has been created
👇 Check out the test result
🔗 https://ABC.atlassian.net/browse/SNP-4579

It would need to be formatted in a way that individual data points ca be pragmatically fetched.

Describe the solution you'd like This way we can use this data to provide additional details to other reporters like https://github.com/ryanrosello-og/playwright-slack-report#readme. For example it would be nice to have a Slack message posted in a channel that says something like "Test Run Completed with 113 Failed 0 Passed. Check out the test results at 🔗 https://ABC.atlassian.net/browse/SNP-4579"

Describe alternatives you've considered None, as I do not think there is a way to get the data from this report console logs.

Additional context None.

Niklas-Back commented 1 month ago

Perhapps in a json file? E.g. runresult.json? { "browsers": "Setup, Webkit", "testPlan": "SNP-3674", "testDuration": "3h 32m", "testsRan": "113 (including reruns)", "testsPassed": "0", "testsFailed": "113", "flakyTests": "0", "skippedTests": "0" }

Or in CTRF? https://medium.com/@ma11hewthomas/its-finally-here-a-json-test-results-data-format-f485b77bbdbc

sn-sstone commented 1 month ago

yea I think a json file would be perfect.

Niklas-Back commented 3 weeks ago

A will add it in the next release

NiklasBack commented 2 weeks ago

I'm thinking that I should timestamp the file, e.g. runresult-20241111155324.json. Should I perhaps put a flag in the configfile e.g. writeSummary: true or just write it at every run?

sn-sstone commented 1 week ago

I think having a way to turn it on/off would be great. I'm not sure if it needs a timestamp, I'll leave that up to you. However Looking at playwright-report and test-results none of those have a time stamp.

NiklasBack commented 1 week ago

I will try it out in our pipeline before submitting it

NiklasBack commented 5 days ago

Closed and Released with: https://github.com/inluxc/playwright-xray/releases/tag/0.6.6

inluxc / playwright-xray

Store PlayRight XRay Results #76