search

microsoft / playwright-testing-service

MIT License
88 stars 14 forks source link

NumberOfTestWorkers not working when using Playwright .Net and Nunit with Microsoft Playwright Testing Service #109

Closed uday2029 closed 5 months ago

uday2029 commented 6 months ago

We are attempting to run this sample test with Microsoft playwright testing service : https://github.com/microsoft/playwright-testing-service/blob/main/samples/.NET/NUnit/README.md

However Microsoft playwright only shows 1 worker every time, even though we have given it as 3.

Here is our runsettings file:

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
    <!-- NUnit adapter -->
    <NUnit>
        <NumberOfTestWorkers>3</NumberOfTestWorkers>
    </NUnit>
    <!-- General run configuration -->
    <RunConfiguration>
        <EnvironmentVariables>
            <PLAYWRIGHT_SERVICE_ACCESS_TOKEN>my access token</PLAYWRIGHT_SERVICE_ACCESS_TOKEN>
            <PLAYWRIGHT_SERVICE_URL>wss://eastus.api.playwright.microsoft.com/api/authorize/connectSession</PLAYWRIGHT_SERVICE_URL>
            <DEBUG>pw:api</DEBUG>
        </EnvironmentVariables>
    </RunConfiguration>
    <!-- Playwright -->
    <Playwright>
        <BrowserName>chromium</BrowserName>
        <ExpectTimeout>10000</ExpectTimeout>
        <LaunchOptions>
            <Headless>true</Headless>
            <Channel>chromium</Channel>
        </LaunchOptions>
    </Playwright>
</RunSettings

image

we are not seeing any issues when other team is using Typescript with number of workers.

uday2029 commented 6 months ago

Here are the debug logs: There is no mention of NumberOfTestWorkers , any clues will be appreciated.

2024-05-20T14:36:26.843Z pw:api navigating to "https://playwright.dev/", waiting until "load"
2024-05-20T14:36:30.010Z pw:api   "commit" event fired
2024-05-20T14:36:30.010Z pw:api   navigated to "https://playwright.dev/"
2024-05-20T14:36:30.355Z pw:api   "domcontentloaded" event fired
2024-05-20T14:36:30.389Z pw:api   "load" event fired
2024-05-20T14:36:30.416Z pw:api PageAssertions.ToHaveTitleAsync with timeout 10000ms
2024-05-20T14:36:30.420Z pw:api waiting for Locator(":root")
2024-05-20T14:36:30.499Z pw:api   locator resolved to <html lang="en" dir="ltr" data-theme="light" data-has-…>…</html>
2024-05-20T14:36:30.514Z pw:api LocatorAssertions.ToHaveAttributeAsync with timeout 10000ms
2024-05-20T14:36:30.516Z pw:api waiting for GetByRole(AriaRole.Link, new() { Name = "Get started" })
2024-05-20T14:36:30.528Z pw:api   locator resolved to <a href="/docs/intro" class="getStarted_Sjon">Get started</a>
2024-05-20T14:36:30.533Z pw:api waiting for GetByRole(AriaRole.Link, new() { Name = "Get started" })
2024-05-20T14:36:30.544Z pw:api   locator resolved to <a href="/docs/intro" class="getStarted_Sjon">Get started</a>
2024-05-20T14:36:30.549Z pw:api attempting click action
2024-05-20T14:36:30.550Z pw:api   waiting for element to be visible, enabled and stable
2024-05-20T14:36:30.573Z pw:api   element is visible, enabled and stable
2024-05-20T14:36:30.573Z pw:api   scrolling into view if needed
2024-05-20T14:36:30.574Z pw:api   done scrolling
2024-05-20T14:36:30.580Z pw:api   performing click action
2024-05-20T14:36:30.603Z pw:api   navigated to "https://playwright.dev/docs/intro"
2024-05-20T14:36:30.616Z pw:api   click action done
2024-05-20T14:36:30.616Z pw:api   waiting for scheduled navigations to finish
2024-05-20T14:36:30.621Z pw:api   navigations have finished
2024-05-20T14:36:30.628Z pw:api PageAssertions.ToHaveURLAsync with timeout 10000ms
2024-05-20T14:36:30.629Z pw:api waiting for Locator(":root")
2024-05-20T14:36:30.640Z pw:api   locator resolved to <html lang="en" dir="ltr" data-theme="light" data-has-…>…</html>
Sid200026 commented 6 months ago

Hi @uday2029

The sample NUnit project provided only contains 1 test file. As per Test Runners | Playwright .NET | Running NUnit tests in Parallel

By default, NUnit will run all test files in parallel, while running tests inside each file sequentially (ParallelScope.Self)

Since the project only contains 1 test file, that maximum number of workers that will be used is 1. If you increase the number of test files, you can observe more parallel workers being used.

Sid200026 commented 6 months ago

Hi @uday2029, let me know if you were able to run tests with multiple parallel workers in NUnit as per the previous suggestion.

puagarwa commented 5 months ago

HI @uday2029 , closing the issue for now, feel free to open it if you need further help.