This adds some checks to the playwright tests to make them fail faster in a few situations.
The premium user in E2E_TEST_ACCOUNT_PREMIUM has to choose a subdomain for the premium user tests to pass. If the "choose a subdomain" banner is displayed on the dashboard, fail the generatePremiumDomainMask test early.
When generating multiple email masks, the generateMask routine waited for any mask cards to be on the page. This means that it would rapidly fire requests for the second and above masks without waiting for the page to show the new mask. This was mitigated with a 1.5 to 2.5 second wait time after adding a mask. The new code:
Counts the mask cards before the request
Waits for either the mask card count to increase by one, or for the "Mask creation failed" banner to be displayed
If the "Mask creation failed" banner is displayed, the test fails early, and suggests rate limiting may be the cause.
The test verifies that exactly one new mask card is present. The react-aria 3.34.3 update resulted in 2 cards, this might catch that early. (PR #5021)
The test now waits .5 to 1.0 seconds, since these checks add about 2 seconds in the happy case. It is possible that no wait is needed now.
When checking for navigation to the premium upsell page, use expect(page).toHaveURL("/premium/#pricing"), which will retry until the test timeout. This makes slow browsers (firefox) a bit more reliable.
It also updates the e2e README with explicit commands for debugging and developing playwright tests, and starts a debugging section.
This adds some checks to the playwright tests to make them fail faster in a few situations.
E2E_TEST_ACCOUNT_PREMIUMhas to choose a subdomain for the premium user tests to pass. If the "choose a subdomain" banner is displayed on the dashboard, fail thegeneratePremiumDomainMasktest early.generateMaskroutine waited for any mask cards to be on the page. This means that it would rapidly fire requests for the second and above masks without waiting for the page to show the new mask. This was mitigated with a 1.5 to 2.5 second wait time after adding a mask. The new code:react-aria 3.34.3update resulted in 2 cards, this might catch that early. (PR #5021)expect(page).toHaveURL("/premium/#pricing"), which will retry until the test timeout. This makes slow browsers (firefox) a bit more reliable.It also updates the e2e README with explicit commands for debugging and developing playwright tests, and starts a debugging section.