microsoft / playwright-testing-service

MIT License
70 stars 10 forks source link

Quota increase request #112

Open lliang87 opened 1 week ago

lliang87 commented 1 week ago

Quota Increase Request

Quota to Increase:

[Specify which quota you would like to increase]

Business Justification: We have a large number of growing e2e tests that are run automatically when PRs are made or updated. We are already encountering an error with connection timeouts to the service as mentioned in this issue which we suspect is because we are hitting the current quota of 50. At the moment we have 5 (soon to be 6) projects' test suites containing an average of 28 tests running in parallel. As we have a fast and frequent turnaround of PRs we want to minimize the time taken for our e2e tests to run as they are required to complete successfully for a merge to occur, hence running all the tests in parallel. We predict that each project's test suite will grow to about an average of 50 tests in which case each run would need about 250 workers in parallel to accommodate this. Additionally, since we have multiple PRs active at a time this could mean 3 - 4 runs simultaneously which is why we are requesting for significantly more workers.

Requested New Quota Limit: We would like to request for 1000 workers per workspace.

Additional Information:

[Include any additional information that may be relevant to your request, such as project deadlines, the impact of the quota increase on your work, etc.]

Setup Information:

Disclaimer:

Please note that while we will do our best to accommodate your quota extension request, we may not be able to fulfill every request due to resource constraints. We will review your request and provide a response as soon as possible.

vvs11 commented 1 week ago

Hi @lliang87 Thanks for raising this request. We are working through our systems to accommodate this request. Meanwhile, can we set up a call to understand more about how you are using the service and your experience with it? Your feedback will help us improve the product. Please send us an email to playwrighttesting@microsoft.com and I will send you a meeting invite. Thanks

vvs11 commented 3 days ago

Hi @lliang87, We have increased the quota for your workspace to 250. Please try it out along with other solutions we discussed. You can also go through this doc: https://learn.microsoft.com/en-us/azure/playwright-testing/concept-determine-optimal-configuration to understand more about how to optimize your test suite.