[BUG] - Playwright Tests are flaky

[nkaretnikov]

Describe the bug

I've been told tests are failing for unrelated reason on this PR: https://github.com/conda-incubator/conda-store-ui/pull/321 Chuck also saw this. Based on what I'm told, the issue goes away after you re-run the tests multiple times, which I'm trying to do on the above PR, let's see if it helps.

Expected behavior

Tests pass.

How to Reproduce the problem?

Run the tests.

Output

No response

Versions and dependencies used.

No response

Anything else?

No response

[nkaretnikov]

https://github.com/conda-incubator/conda-store-ui/actions/runs/6746847691/job/18341697088

>       assert not page.get_by_role("button", name=env_name).is_visible()
E       assert not True
E        +  where True = <bound method Locator.is_visible of <Locator frame=<Frame name= url='http://localhost:8080/'> selector='internal:role=button[name="test_env_217"i]'>>()
E        +    where <bound method Locator.is_visible of <Locator frame=<Frame name= url='http://localhost:8080/'> selector='internal:role=button[name="test_env_217"i]'>> = <Locator frame=<Frame name= url='http://localhost:8080/'> selector='internal:role=button[name="test_env_217"i]'>.is_visible
E        +      where <Locator frame=<Frame name= url='http://localhost:8080/'> selector='internal:role=button[name="test_env_217"i]'> = <bound method Page.get_by_role of <Page url='http://localhost:8080/'>>('button', name='test_env_217')
E        +        where <bound method Page.get_by_role of <Page url='http://localhost:8080/'>> = <Page url='http://localhost:8080/'>.get_by_role

test/playwright/test_ux.py:231: AssertionError
=========================== short test summary info ============================
FAILED test/playwright/test_ux.py::test_integration[chromium] - assert not True
 +  where True = <bound method Locator.is_visible of <Locator frame=<Frame name= url='http://localhost:8080/'> selector='internal:role=button[name="test_env_217"i]'>>()
 +    where <bound method Locator.is_visible of <Locator frame=<Frame name= url='http://localhost:8080/'> selector='internal:role=button[name="test_env_217"i]'>> = <Locator frame=<Frame name= url='http://localhost:8080/'> selector='internal:role=button[name="test_env_217"i]'>.is_visible
 +      where <Locator frame=<Frame name= url='http://localhost:8080/'> selector='internal:role=button[name="test_env_217"i]'> = <bound method Page.get_by_role of <Page url='http://localhost:8080/'>>('button', name='test_env_217')
 +        where <bound method Page.get_by_role of <Page url='http://localhost:8080/'>> = <Page url='http://localhost:8080/'>.get_by_role
======================== 1 failed in 144.69s (0:02:24) =========================
Error: Process completed with exit code 1.

[nkaretnikov]

I think we just need to insert a 5 second sleep before this:

>       assert not page.get_by_role("button", name=env_name).is_visible()

I've seen web UIs which take a while to load. I think we might be checking too fast.