search

microsoft / playwright

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
https://playwright.dev
Apache License 2.0
66.8k stars 3.66k forks source link

[BUG] Endless running test on selenium hub + nodes (session-timeout) #23651

Open taddeuz opened 1 year ago

taddeuz commented 1 year ago

System info

Steps

Expected

I think it's due to the communication between Playwright and Selenium. If Selenium exits the test after the "session-timeout", then Playwright should abort the test.

Actual

Playwright continues test even selenium has ended the session.

Description I've seen the problem more than once. It's difficult to reproduce. The behavior is as follows: The test starts and runs interactions. At some point the test gets stuck, Playwright does not abort the test (added the Playwright timeouts, 30000ms, and in most cases the tests will be aborted, if no element found). The Selenium node ends the session after the selenium session timeout, e.g. 5 minutes ("session-timeout" is configurable when starting the node). After Selenium Node has stopped the session, Playwright doesn't seem to get any feedback that the session has ended and the test continues indefinitely.

The problem occurs randomly. I cannot provide any test code, sometimes any test gets stuck while performing an interaction.

I am starting the hub and nodes with the following configurations:

start java -jar selenium-server-4.8.3.jar hub start java -jar selenium-server-4.8.3.jar node --hub --session-timeout 300

I am starting 4 nodes.

taddeuz commented 1 year ago

Additional information: I can reproduce this issue only when I start a hub and multiple nodes.

When I start hub and note on one single VM, I cannot reproduce the issue.

On 4 different VMs (Selenium Node started on each VM), I ran sporadically into the issues, that any test gets stuck.

I think the main problem is related to the "session-timeout" of selenium. Check with the selenium session timeout, for example 1 minute and you test ran longer than 1 minute. After Selenium has stopped the session after 1 minute, Playwright doesn't seem to get any feedback that the session has ended and the test continues indefinitely.