search

microsoft / playwright-testing-service

MIT License
70 stars 10 forks source link

[BUG]:[Instability of tests when running in 50 workers.] #114

Open sleepmac opened 1 week ago

sleepmac commented 1 week ago

Describe the bug Hi. We are facing the problem of running tests in 50 workers. Earlier we ran tests on a virtual machine in 12 workers. CPU load was 60-80%. When we run tests in the service in 50 workers, the CPU load is 10-20%. As we can see on the graph. But when we run the tests through the service, they are very unstable. The application slows down, windows take a long time to open, images are rendered with a long delay. The tests are run for three projects Desktop Chrome, Mobile Chrome, Mobile Webkit. What can be the problem and how can we solve it?

Expected behavior The tests pass.

Screenshots 26

Setup information (please complete the following information):

vvs11 commented 1 week ago

Hi @sleepmac Thanks for reaching out. Could you share more information to help us support you better? 1, What is the configuration of the machine that you are using to run the tests?

  1. How are you measuring the load times of the application that you are testing?
  2. Where is the application under test hosted? Is it public/private?
  3. If private, are you using the expose-network setting provided by Playwright?
sleepmac commented 6 days ago

Hi @vvs11

  1. 12 CPU, 128gb RAM
  2. Private. We've added your ip to the whitelist 98.71.172.224/28
  3. No, we don't use this setting.
sleepmac commented 4 days ago

@vvs11 Hi. Any news, updates?

puagarwa commented 3 days ago

Hi @sleepmac Can you help us provide more info into unstable test and application slowness issue you are facing.

In general, here is possible reasons

From the above description, where you mentioned around webapp slowness it seems webapp under test is not behaving good under 50 parallelism load.