Open imanish003 opened 1 year ago
Great write up @imanish003!. I agree that using JS to create test data will benefit us in the long run.
The new
.sql
file would only need to be generated when our fixture data changes.
I think there are more cases we may need to update the .sql
file:
Those screen can break out tests, and we shouldn't check for their existence. Instead, we should ensure they won't show up (by regenerate the sql file in this case).
Upon migration, fixture-data.js would be used to set up the test environment and subsequently export a woocommerce.sql file. This .sql file would then be imported into the E2E test environment. This method will help reduce the test setup time significantly. The new .sql file would only need to be generated when our fixture data changes.
As a follow-up to https://github.com/woocommerce/woocommerce/pull/46125, we've briefly discussed using a pre-existing db dump instead of generating the environment each time. Since the import logic is already there, it should be straightforward to implement. Indeed it would make things much faster! 🙌
Regarding the test data transition to fixture-data.js
, I think it's a good idea that would help write more deterministic tests, especially in situations where we need to know the exact number of products to assert against. This could be implemented in a Playwright Fixture convention, making the data available on-demand for each test without explicitly importing it.
IMO, the two issues above can also be dealt with separately as they have no direct dependency. What do you think, @imanish003?
cc: @gigitux
Yeah, I agree with you, @WunderBart. We can handle the two issues separately 💯
This issue is result of following GH discussion: https://github.com/woocommerce/woocommerce-blocks/discussions/10099
As we strive to streamline our E2E testing setup, we propose to establish
fixture-data.js
as the central place for test data in our Playwright E2E tests.The main objectives of this enhancement include:
test-env-setup.sh
tofixture-data.js
: Thetest-env-setup.sh
script currently performs several tasks including emptying the site, adding attributes, importing sample products, regenerating product lookup tables, creating pages and posts, and setting up payment methods. We aim to shift these responsibilities tofixture-data.js
.fixture-data.js
with Additional Test Data: Alongside handling the tasks currently executed bytest-env-setup.sh
,fixture-data.js
would also store the extra data needed for Playwright E2E tests.fixture-data.js
as the Single Source of Truth for Test Setup: Upon migration,fixture-data.js
would be used to set up the test environment and subsequently export awoocommerce.sql
file. This.sql
file would then be imported into the E2E test environment. This method will help reduce the test setup time significantly. The new.sql
file would only need to be generated when our fixture data changes.For more info: https://github.com/woocommerce/woocommerce-blocks/discussions/10099