Open RF2023 opened 2 months ago
puagarwa
commented
2 months ago Hi @RF2023 The stacktrace suggest error is coming from qase module and it doesnt seems like coming from playwright service. I would suggest you to check with QASE team. If you still think this is related to MS playwright service, let me know why.
RF2023
commented
1 month ago I opened a ticket with QASE, i will keep you posted.
puagarwa
commented
1 month ago Closing the issue for now, feel free to reopen it if you still think its related to playwright service.
RF2023
commented
1 month ago So we are clearly seeing an issue when we use playwright.config.ts vs playwright.service.config.ts
latest error, it starts of hitting the qase API we get a notification saying qase run is created but , can you still confirm if this is not relating to the service and is a QASE issue? I have a open ticket with them as well.
`qase: Current PID: 2695
Running 32 tests using 10 workers
qase: Project ***** exists qase: Using run 142 to publish test results`
`node:internal/fs/utils:699
throw new ERR_INVALID_ARG_TYPE(propName, ['string', 'Buffer', 'URL'], path);
^
TypeError: The "path" argument must be of type string or an instance of Buffer or URL. Received undefined
at new ReadStream (node:internal/fs/streams:185:5)
at Object.createReadStream (node:fs:3076:10)
at PlaywrightReporter.<anonymous> (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/src/index.ts:390:30)
at step (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:64:23)
at Object.next (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:45:53)
at /runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:39:71
at new Promise (<anonymous>)
at __awaiter (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:35:12)
at /runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/src/index.ts:389:57
at Array.map (<anonymous>)
at PlaywrightReporter.<anonymous> (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/src/index.ts:389:36)
at step (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:64:23)
at Object.next (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:45:53)
at /runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:39:71
at new Promise (<anonymous>)
at __awaiter (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:35:12)
at PlaywrightReporter.uploadAttachments (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:445:16)
at PlaywrightReporter.<anonymous> (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/src/index.ts:233:43)
at step (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:64:23)
at Object.next (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:45:53)
at /runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:39:71
at new Promise (<anonymous>)
at __awaiter (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:35:12)
at PlaywrightReporter.onTestEnd (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:247:16)
at ReporterV2Wrapper.onTestEnd (/runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/reporters/reporterV2.js:87:150)
at /runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/reporters/multiplexer.js:47:65
at wrap (/runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/reporters/multiplexer.js:86:5)
at Multiplexer.onTestEnd (/runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/reporters/multiplexer.js:47:45)
at InternalReporter.onTestEnd (/runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/reporters/internalReporter.js:62:20)
at JobDispatcher._reportTestEnd (/runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/runner/dispatcher.js:518:20)
at JobDispatcher._onTestEnd (/runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/runner/dispatcher.js:284:10)
at WorkerHost.emit (node:events:514:28)
at ChildProcess.<anonymous> (/runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/runner/processHost.js:105:16)
at ChildProcess.emit (node:events:514:28)
at emit (node:internal/child_process:951:14)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
code: 'ERR_INVALID_ARG_TYPE'
}
Node.js v20.10.0
Playwright tests failed with exit code 1`HI @RF2023 , Even though the above log clearly pointing at issue in Qase package but lets do below to further understanding if MPT service has any role here.
Basically we are trying to find of we have issue in service.config or service+Qase use case.
RF2023
commented
1 month ago Thats exactly what i was thinking to isolate whether its just qase or service+case. For now its looking like the later, but let me experiment with yours and get back to you. Thank you !!
On Fri, May 17, 2024 at 3:19 AM Puneet Agarwal @.***> wrote:
HI @RF2023 https://github.com/RF2023 , Even though the above log clearly pointing at issue in Qase package but lets do below to further understanding if MPT service has any role here.
- playwright.service.config inherit playwright.config and override few fields where one is them is essential for service integration which is connectOption, others are optional like ignoreSnapshots, snapshotPathTemplate, please comment these two and try once, if works then all good.
- We could also try to disable service via this change https://github.com/microsoft/playwright-testing-service/pull/108/files
- if issue persist then service.config has issue and we can work it out but service itself doesnt have any issue.
- if issue goes away, then we need to deep dive to understand qase dependency.
Basically we are trying to find of we have issue in service.config or service+Qase use case.
— Reply to this email directly, view it on GitHub https://github.com/microsoft/playwright-testing-service/issues/106#issuecomment-2116911270, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZ6GIWB6YF7AK2TREVVEJGLZCWVQJAVCNFSM6AAAAABHCJATDWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJWHEYTCMRXGA . You are receiving this because you were mentioned.Message ID: @.***>
-- Regards,
Shravan Kumar Eetha Sr QA Engineer
Lively, Inc. | www.livelyme.com
RF2023
commented
1 month ago On Fri, May 17, 2024 at 3:19 AM Puneet Agarwal @.***> wrote:
HI @RF2023 https://github.com/RF2023 , Even though the above log clearly pointing at issue in Qase package but lets do below to further understanding if MPT service has any role here.
- playwright.service.config inherit playwright.config and override few fields where one is them is essential for service integration which is connectOption, others are optional like ignoreSnapshots, snapshotPathTemplate, please comment these two and try once, if works then all good.
- We could also try to disable service via this change https://github.com/microsoft/playwright-testing-service/pull/108/files
- if issue persist then service.config has issue and we can work it out but service itself doesnt have any issue.
- if issue goes away, then we need to deep dive to understand qase dependency.
Basically we are trying to find of we have issue in service.config or service+Qase use case.
— Reply to this email directly, view it on GitHub https://github.com/microsoft/playwright-testing-service/issues/106#issuecomment-2116911270, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZ6GIWB6YF7AK2TREVVEJGLZCWVQJAVCNFSM6AAAAABHCJATDWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJWHEYTCMRXGA . You are receiving this because you were mentioned.Message ID: @.***>
-- Regards,
Shravan Kumar Eetha Sr QA Engineer
Lively, Inc. | www.livelyme.com
puagarwa
commented
1 month ago HI @RF2023, can you please try my other suggestion as well We could also try to disable service via this change
RF2023
commented
1 month ago So when you say disable service, the tests are not running on service unless we specify the service file. Did i understand that correctly? Do you want me to still make the changes suggested and use the service config file? This is how our file which triggers the tests to qase looks test: name: ** uses: ./.github/workflows/auto.yml with: environment: localhost config_file: playwright.service.config.ts tag: ***** send_to_qase: true
On Fri, May 17, 2024 at 2:54 PM Puneet Agarwal @.***> wrote:
HI @RF2023 https://github.com/RF2023, can you please try my other suggestion as well We could also try to disable service via this change https://github.com/microsoft/playwright-testing-service/pull/108/files
- if issue persist then service.config has issue and we can work it out but service itself doesnt have any issue.
- if issue goes away, then we need to deep dive to understand qase dependency.
— Reply to this email directly, view it on GitHub https://github.com/microsoft/playwright-testing-service/issues/106#issuecomment-2118205445, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZ6GIWDSB5BEWNTS4CWL7QDZCZG5JAVCNFSM6AAAAABHCJATDWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJYGIYDKNBUGU . You are receiving this because you were mentioned.Message ID: @.***>
-- Regards,
Shravan Kumar Eetha Sr QA Engineer
Lively, Inc. | www.livelyme.com
Sid200026
commented
1 month ago Hi @RF2023
So when you say disable service, the tests are not running on service unless we specify the service file. Did i understand that correctly? Do you want me to still make the changes suggested and use the service config file?
Since running with no service using playwright.config.ts file is already working, we want you to run with service disabled but using the playwright.service.config.ts file. This can be achieved by not passing the connectOptions parameter - Reference PR - env variable to toggle MPT service #108
RF2023
commented
1 month ago playwright test --project=chromium -g @testtag -c playwright.service.config.ts qase: Current PID: 2700 Running 32 tests using 10 workers qase: Project ** exists qase: Using run 150 to publish test results According to the mentioned changes, its working and publishing results without failing. https://github.com/microsoft/playwright-testing-service/pull/108/files did the exact changes.
Sid200026
commented
1 month ago Hi @RF2023,
Could you share the playwright-qase-reporter version that is being used in your project?
I integrated qase to the sample playwright project as per qase playwright integration example and was able to publish tests results to qase while using service config with MPT enabled.
I encountered a similar error when used v1.2.2 and had to upgrade to v2.0.0. There's a fix from the qase team regarding your issue - TypeError: The "path" argument must be of type string or an instance of Buffer or URL. Received undefined at new ReadStream which checks if the path is undefined before creating the read stream.
// v1.2.2
const data = createReadStream(attachment?.path as string);// v2.0.0
if (attachment.file_path) {
data = { name: attachment.file_name, value: createReadStream(attachment.file_path) };
}"playwright-qase-reporter": "^1.2.2", upgrade was breaking our changes the other day before i reopened this ticket. also this is how the previous change looks like
// v1.2.2 const data = createReadStream(attachment?.path as string); does this need to be added??
previous change looks like this
use: {
// Specify the service endpoint.
connectOptions: process.env.ENABLE_MPT_SERVICE === '1' ?{
wsEndpoint: ${process.env.PLAYWRIGHT_SERVICE_URL}?cap=${JSON.stringify({ // Can be 'linux' or 'windows'. os, runId: process.env.PLAYWRIGHT_SERVICE_RUN_ID, })},
timeout: 30000,
headers: {
'x-mpt-access-key': process.env.PLAYWRIGHT_SERVICE_ACCESS_TOKEN!,
},
// Allow service to access the localhost.
exposeNetwork: '
On Tue, May 21, 2024 at 6:44 AM Siddharth Singha Roy < @.***> wrote:
Hi @RF2023 https://github.com/RF2023,
Could you share the playwright-qase-reporter version that is being used in your project?
I integrated qase to the sample playwright project as per qase playwright integration example https://github.com/qase-tms/qase-javascript/tree/main/examples/playwright and was able to publish tests results to qase while using service config with MPT enabled.
I encountered a similar error when used v1.2.2 and had to upgrade to v2.0.0. There's a fix from the qase team regarding your issue - TypeError: The "path" argument must be of type string or an instance of Buffer or URL. Received undefined at new ReadStream which checks if the path is undefined before creating the read stream.
// v1.2.2const data = createReadStream(attachment?.path as string);
// v2.0.0if (attachment.file_path) { data = { name: attachment.file_name, value: createReadStream(attachment.file_path) };}
— Reply to this email directly, view it on GitHub https://github.com/microsoft/playwright-testing-service/issues/106#issuecomment-2122333039, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZ6GIWHLVQ2J3CKCWXCYND3ZDMQPZAVCNFSM6AAAAABHCJATDWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRSGMZTGMBTHE . You are receiving this because you were mentioned.Message ID: @.***>
-- Regards,
Shravan Kumar Eetha Sr QA Engineer
Lively, Inc. | www.livelyme.com
Sid200026
commented
1 month ago does this need to be added??
No, it's a part of the playwright-qase-reporter package. I was just highlighting the line of code (in the playwright-qase-reporter package) which is causing the error you faced.
The only way forward I see is you upgrading playwright-qase-reporter to v2.0.0 which contains the fix.
upgrade was breaking our changes the other day before i reopened this ticket.
When I was upgrading the qase reporter package from v1.2.2 to v2.0.0 locally, I faced upgrade issues as well. I resolved it by following the new Reporter options provided by qase as per - playwright config for Playwright - Qase Integration
previous change looks like this
You can revert the debug changes you did earlier as part of Reference PR - env variable to toggle MPT service #108 since they are no longer required.
RF2023
commented
1 month ago When I was upgrading the qase reporter package from v1.2.2 to v2.0.0 locally, I faced upgrade issues as well. I resolved it by following the new Reporter options provided by qase as per - playwright config for Playwright
let me try the above. Thank you for the prompt response.
On Wed, May 22, 2024 at 10:20 AM Siddharth Singha Roy < @.***> wrote:
does this need to be added??
No, it's a part of the playwright-qase-reporter package. I was just highlighting the line of code (in the playwright-qase-reporter package) which is causing the error you faced.
The only way forward I see is you upgrading playwright-qase-reporter to v2.0.0 which contains the fix.
upgrade was breaking our changes the other day before i reopened this ticket.
When I was upgrading the qase reporter package from v1.2.2 to v2.0.0 locally, I faced upgrade issues as well. I resolved it by following the new Reporter options provided by qase as per - playwright config for Playwright - Qase Integration https://github.com/qase-tms/qase-javascript/blob/main/examples/playwright/playwright.config.js
previous change looks like this
You can revert the debug changes you did earlier as part of Reference PR
- env variable to toggle MPT service #108 https://github.com/microsoft/playwright-testing-service/pull/108/files since they are no longer required.
— Reply to this email directly, view it on GitHub https://github.com/microsoft/playwright-testing-service/issues/106#issuecomment-2124925249, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZ6GIWDO6SVQC2JKWW4ZFODZDSSSDAVCNFSM6AAAAABHCJATDWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRUHEZDKMRUHE . You are receiving this because you were mentioned.Message ID: @.***>
-- Regards,
Shravan Kumar Eetha Sr QA Engineer
Lively, Inc. | www.livelyme.com
RF2023
commented
1 month ago upgraded the playwright-qase-reporter, dont see results being send to qase, followed playwright config for Playwright - Qase Integration https://github.com/qase-tms/qase-javascript/blob/main/examples/playwright/playwright.config.js reverted changes for MPT, its not even hitting QASE API and at the same time its not failing.
'playwright-qase-reporter', { debug: true, basePath: 'https://api.qase.io/v1', runId: process.env., testops: { api: { token: process.env., }, project: process.env.*****, uploadAttachments: true, run: { complete: true, }, }, }, ]); }
On Wed, May 22, 2024 at 10:20 AM Siddharth Singha Roy < @.***> wrote:
does this need to be added??
No, it's a part of the playwright-qase-reporter package. I was just highlighting the line of code (in the playwright-qase-reporter package) which is causing the error you faced.
The only way forward I see is you upgrading playwright-qase-reporter to v2.0.0 which contains the fix.
upgrade was breaking our changes the other day before i reopened this ticket.
When I was upgrading the qase reporter package from v1.2.2 to v2.0.0 locally, I faced upgrade issues as well. I resolved it by following the new Reporter options provided by qase as per - playwright config for Playwright - Qase Integration https://github.com/qase-tms/qase-javascript/blob/main/examples/playwright/playwright.config.js
previous change looks like this
You can revert the debug changes you did earlier as part of Reference PR
- env variable to toggle MPT service #108 https://github.com/microsoft/playwright-testing-service/pull/108/files since they are no longer required.
— Reply to this email directly, view it on GitHub https://github.com/microsoft/playwright-testing-service/issues/106#issuecomment-2124925249, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZ6GIWDO6SVQC2JKWW4ZFODZDSSSDAVCNFSM6AAAAABHCJATDWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRUHEZDKMRUHE . You are receiving this because you were mentioned.Message ID: @.***>
-- Regards,
Shravan Kumar Eetha Sr QA Engineer
Lively, Inc. | www.livelyme.com
RF2023
commented
1 month ago lively-automation@1.0.0 play playwright test --project=chromium -g @testtag -c playwright.config.ts qase: Current PID: **** Running 32 tests using 4 workers qase: Project automation exists qase: Using run 154 to publish test results
works fine when i use the normal file, can you please share the service file that worked for you along with changes made as part of new qase reporter upgrade?
also tried it with this change to service file, doesnt error out but doesnt publish as well
Also on the service file , following is commented out as part of previous suggestion.
// Enable screenshot testing and configure directory with expectations.
// https://learn.microsoft.com/azure/playwright-testing/how-to-configure-visual-comparisons
//ignoreSnapshots: false,
//snapshotPathTemplate: {testDir}/__screenshots__/{testFilePath}/${os}/{arg}{ext},
Sid200026
commented
1 month ago works fine when i use the normal file, can you please share the service file that worked for you along with changes made as part of new qase reporter upgrade?
I used the sample MPT config and just added the qase reporter as per below.
reporter: [
['list'],
[
'playwright-qase-reporter',
{
debug: true,
testops: {
api: {
token: '<>',
},
project: 'DEMO',
uploadAttachments: true,
run: {
complete: true,
},
},
},
],
],I also had to set an environment variable QASE_MODE = testops as per test script in package.json
RF2023
commented
3 weeks ago So with the requested changes of upgrade to playwright qase reporter and making the changes to even env variables, sending results to QASE is working fine with normal config and PW service. The new issue is the allure reports not displaying the QASE ID, i raised it with QASE and they say its not their side of the issue.
[image: Screenshot 2024-06-04 at 2.10.08 PM.png] This is a allure report with QASE ID displayed before the reporter was upgraded. [image: Screenshot 2024-06-04 at 2.12.30 PM.png]
On Thu, May 23, 2024 at 3:07 AM Siddharth Singha Roy < @.***> wrote:
works fine when i use the normal file, can you please share the service file that worked for you along with changes made as part of new qase reporter upgrade?
I used the sample MPT config https://github.com/microsoft/playwright-testing-service/blob/main/samples/get-started/playwright.service.config.ts and just added the qase reporter as per below.
reporter: [ ['list'], [ 'playwright-qase-reporter', { debug: true,
testops: { api: { token: '<>', }, project: 'DEMO', uploadAttachments: true, run: { complete: true, }, }, }, ],],
I also had to set an environment variable QASE_MODE = testops as per test script in package.json https://github.com/qase-tms/qase-javascript/blob/main/examples/playwright/package.json
— Reply to this email directly, view it on GitHub https://github.com/microsoft/playwright-testing-service/issues/106#issuecomment-2126381107, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZ6GIWDD7YQWLKQOONQRWNLZDWIRPAVCNFSM6AAAAABHCJATDWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRWGM4DCMJQG4 . You are receiving this because you were mentioned.Message ID: @.***>
-- Regards,
Shravan Kumar Eetha Sr QA Engineer
Lively, Inc. | www.livelyme.com
puagarwa
commented
3 weeks ago HI @RF2023 We dont have any knowledge about qase reporter, but we could try to bifurcate the issue with playwright service so that you can easily work with qase team on the issue.
Here is what i would suggest, you can make the qase work in your non-service config (playwright.config.ts) and work with qase team to resolve if any issue. Once it works fine then you can share the config with us and we can help you with changes to integrate service, and if you find any issue then we would work with you on fix. if require we can schedule a call for faster resolution, please send mail to playwrighttesting@microsoft.com.
Ideally, our service changes shouldn't interfere with reporter working because both are different worker thread which works independently. so most likely if you are facing issue with qase then issue is either in qase integration or in their package.
vvs11
commented
3 weeks ago Hi @RF2023
We have recently introduced the reporting feature for Microsoft Playwright Testing service. With this feature, you can publish your test results and artifacts to the service and view them in the MPT portal. You'll have all the information needed to diagnose failed tests. The portal also hosts all collected artifacts, including the trace viewer. Integrating this feature with your existing test suite is straightforward and easy.
You can read more about it here: https://aka.ms/mpt/reporting-announce
Currently, the feature is invite-only preview. You can sign up using this link: https://aka.ms/mpt/reporting-signup
Thanks
RF2023
commented
3 weeks ago Puneet, thank you for the response. This is what i will be doing going forward
Let me know what exact information you need from me once i have it back working on the config file. Thank you for your time !
On Wed, Jun 5, 2024 at 12:49 AM Puneet Agarwal @.***> wrote:
HI @RF2023 https://github.com/RF2023 We dont have any knowledge about qase reporter, but we could try to bifurcate the issue with playwright service so that you can easily work with qase team on the issue.
Here is what i would suggest, you can make the qase work in your non-service config (playwright.config.ts) and work with qase team to resolve if any issue. Once it works fine then you can share the config with us and we can help you with changes to integrate service, and if you find any issue then we would work with you on fix. if require we can schedule a call for faster resolution, please send mail to @.***
Ideally, our service changes shouldn't interfere with reporter working because both are different worker thread which works independently. so most likely if you are facing issue with qase then issue is either in qase integration or in their package.
— Reply to this email directly, view it on GitHub https://github.com/microsoft/playwright-testing-service/issues/106#issuecomment-2148848430, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZ6GIWA5N5NFVRDR5UY5LG3ZF2KFZAVCNFSM6AAAAABHCJATDWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBYHA2DQNBTGA . You are receiving this because you were mentioned.Message ID: @.***>
-- Regards,
Shravan Kumar Eetha Sr QA Engineer
Lively, Inc. | www.livelyme.com
RF2023
commented
3 weeks ago Absolutely, i was thinking about it and will do it after i talk to the team.
On Wed, Jun 5, 2024 at 2:58 AM Vansh Vardhan Singh @.***> wrote:
Hi @RF2023 https://github.com/RF2023 We have recently announced the reporting feature for Microsoft Playwright Testing service. This feature allows you to publish your test results and artifacts directly to the service, making them accessible in the MPT portal. Here, you'll find all the information needed to diagnose any failed tests. The portal also includes a trace viewer and hosts all collected artifacts for each test. Integrating this feature into your existing test suite is simple and straightforward.
You can read more about it here: https://aka.ms/mpt/reporting-announce
Currently, the feature is invite-only preview. You can sign up using this link: https://aka.ms/mpt/reporting-signup
— Reply to this email directly, view it on GitHub https://github.com/microsoft/playwright-testing-service/issues/106#issuecomment-2149024449, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZ6GIWG2UGL4BK5KCCO5Y4TZF2ZIVAVCNFSM6AAAAABHCJATDWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBZGAZDINBUHE . You are receiving this because you were mentioned.Message ID: @.***>
-- Regards,
Shravan Kumar Eetha Sr QA Engineer
Lively, Inc. | www.livelyme.com
puagarwa
commented
3 weeks ago Puneet, thank you for the response. This is what i will be doing going forward 1. Downgrade playwright-qase-reporter back to 1.2.2 from 2.0.0 2. Run tests with config file and make sure its working as expected. 3. Run it with service file as well (wont work) Let me know what exact information you need from me once i have it back working on the config file. Thank you for your time ! … On Wed, Jun 5, 2024 at 12:49 AM Puneet Agarwal @.> wrote: HI @RF2023 https://github.com/RF2023 We dont have any knowledge about qase reporter, but we could try to bifurcate the issue with playwright service so that you can easily work with qase team on the issue. Here is what i would suggest, you can make the qase work in your non-service config (playwright.config.ts) and work with qase team to resolve if any issue. Once it works fine then you can share the config with us and we can help you with changes to integrate service, and if you find any issue then we would work with you on fix. if require we can schedule a call for faster resolution, please send mail to @. Ideally, our service changes shouldn't interfere with reporter working because both are different worker thread which works independently. so most likely if you are facing issue with qase then issue is either in qase integration or in their package. — Reply to this email directly, view it on GitHub <#106 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZ6GIWA5N5NFVRDR5UY5LG3ZF2KFZAVCNFSM6AAAAABHCJATDWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBYHA2DQNBTGA . You are receiving this because you were mentioned.Message ID: @.***> -- Regards, Shravan Kumar Eetha Sr QA Engineer Lively, Inc. | www.livelyme.com https://livelyme.com/
Hi @RF2023, I actually think you can make qase latest version work without our service and then share your config file with us and we will integrate service. As my team mate earlier mentioned there was known issue in thier old package https://github.com/microsoft/playwright-testing-service/issues/106#issuecomment-2122333039
RF2023
commented
3 weeks ago Problem is the latest version of playwright qase reporter is not showing the QASE ID in the report both with the service and normal config file. Thats why i was planning to downgrade to 1.2.2. I can still share what we have currently, what is the exact piece you guys are looking for , so that i can share the masked code snippet?
puagarwa
commented
3 weeks ago i see, i would advise to work with them to resolve issue with latest version.
But if you still wants to work with old qase version, then we could try integrating service in your default config file via this kind of change https://github.com/microsoft/playwright-testing-service/pull/110 ,
We can try to find some time for faster resolution over call , please send mail at playwrighttesting@microsoft.com
RF2023
commented
3 weeks ago Currently in the config file, this is what we have which is working for both after upgrading playwright qase reporter to 2.0
This is the new suggested format which is working with both files, but at the same time QASE ID at the end is missing for service and normal file. QASE ID is displayed for both
if (process.env.QASE_REPORT) { reporters.push([ 'playwright-qase-reporter', { basePath: 'https://api.qase.io/v1', runId: process.env.QASE_TESTOPS_RUN_ID, testops: { api: { token: process.env.QASE_API_TOKEN, }, project: process.env.QASE_TESTOPS_PROJECT, uploadAttachments: true, run: { complete: true, }, }, }, ]); }
Old Format: Works with both files, but send_to_qase = true is selected keeps failing on service and normal file.
if (process.env.QASE_REPORT) { reporters.push([ 'playwright-qase-reporter', { basePath: 'https://api.qase.io/v1', apiToken: process.env.QASE_API_TOKEN, projectCode: process.env.QASE_PROJECT_CODE, runId: process.env.QASE_RUN_ID, runComplete: false, logging: true, uploadAttachments: true, }, ]); }
RF2023
commented
3 weeks ago i see, i would advise to work with them to resolve issue with latest version.
But if you still wants to work with old qase version, then we could try integrating service in your default config file via this kind of change #110 ,
- first get your suite working with non service config
- then modify non service config like this PR
- Run it and report the result
We can try to find some time for faster resolution over call , please send mail at playwrighttesting@microsoft.com
i will send you an invite shortly.
RF2023
commented
3 weeks ago Ok downgraded to v1.2.2 for qase reporter. Reverted back to old format all the changes and also made the playwright.config.ts as mentioned in the above PR. It was running on service and i am assuming with the PR change suggested we cant use the normal config file anymore as everything will only run on service?
On Wed, Jun 5, 2024 at 2:32 PM Puneet Agarwal @.***> wrote:
i see, i would advise to work with them to resolve issue with latest version.
But if you still wants to work with old qase version, then we could try integrating service in your default config file via this kind of change
110 https://github.com/microsoft/playwright-testing-service/pull/110 ,
- first get your suite working with non service config
- then modify non service config like this PR
- Run it and report the result
We can try to find some time for faster resolution over call , please send mail at @.***
— Reply to this email directly, view it on GitHub https://github.com/microsoft/playwright-testing-service/issues/106#issuecomment-2150702262, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZ6GIWASIRJR57XJF737CX3ZF5KR7AVCNFSM6AAAAABHCJATDWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNJQG4YDEMRWGI . You are receiving this because you were mentioned.Message ID: @.***>
-- Regards,
Shravan Kumar Eetha Sr QA Engineer
Lively, Inc. | www.livelyme.com
puagarwa
commented
3 weeks ago @RF2023 Great to hear things worked this way. You could still switch between service and non service mode, if you could add condition env variable like done in this PR https://github.com/microsoft/playwright-testing-service/pull/108/files , Let me know if any doubt here.
RF2023
commented
2 weeks ago So we are good with QASE ID being displayed in the report, but
puagarwa
commented
2 weeks ago HI @RF2023, Both these questions are related to qase and we dont have any knowledge about the qase hence it would be better if you ask them with qase team . I would be happy to help you with any issues/questions in MPT service.
Describe the bug Enabling qase on our build.yml file and running on MS playwright service is throwing an error.
To Reproduce Steps to reproduce the behavior:
Expected behavior Test results should be sent to qase
Screenshots If applicable, add screenshots to help explain your problem.
Setup information (please complete the following information): Azure Subscription Id:798f112d-0c47-431d-9b57-eeae586eb023 Workspace Name: Playwright Service trace Id (if available): Log snippet (if available): Browser on which tests are executed (if applicable): chromium Time range (in PST) when the issue was experienced: Multiple times Playwright Version: 1.42.1 Client machine OS: MAC OS requested on Service: linux
Additional context We have verified this after upgrading our node version as well, based on another ticket but the error still persists. Due to this we are unable to enable qase for MS Playwright service.
ErrorCaptureStackTrace(err); ^ TypeError: The "path" argument must be of type string or an instance of Buffer or URL. Received undefined at new ReadStream (node:internal/fs/streams:174:5) at Object.createReadStream (node:fs:2916:10) at PlaywrightReporter.<anonymous> (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/src/index.ts:388:30) at step (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:64:23) at Object.next (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:45:53) at /runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:39:71 at new Promise (<anonymous>) at __awaiter (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:35:12) at /runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/src/index.ts:387:57 at Array.map (<anonymous>) at PlaywrightReporter.<anonymous> (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/src/index.ts:387:36) at step (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:64:23) at Object.next (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:45:53) at /runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:39:71 at new Promise (<anonymous>) at __awaiter (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:35:12) at PlaywrightReporter.uploadAttachments (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:443:16) at PlaywrightReporter.<anonymous> (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/src/index.ts:231:43) at step (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:64:23) at Object.next (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:45:53) at /runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:39:71 at new Promise (<anonymous>) at __awaiter (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:35:12) at PlaywrightReporter.onTestEnd (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:245:16) at ReporterV2Wrapper.onTestEnd (/runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/reporters/reporterV2.js:87:158) at /runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/reporters/multiplexer.js:47:65 at wrap (/runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/reporters/multiplexer.js:86:5) at Multiplexer.onTestEnd (/runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/reporters/multiplexer.js:47:45) at InternalReporter.onTestEnd (/runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/reporters/internalReporter.js:62:20) at JobDispatcher._reportTestEnd (/runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/runner/dispatcher.js:516:20) at JobDispatcher._onTestEnd (/runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/runner/dispatcher.js:288:10) at WorkerHost.emit (node:events:513:28) at ChildProcess.<anonymous> (/runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/runner/processHost.js:105:16) at ChildProcess.emit (node:events:513:28) at emit (node:internal/child_process:946:14) at processTicksAndRejections (node:internal/process/task_queues:84:21) { code: 'ERR_INVALID_ARG_TYPE' }