10up / ads-txt

Ads.txt Manager for WordPress: Manage your ads.txt and app-ads.txt files in the WordPress dashboard
https://wordpress.org/plugins/ads-txt/
GNU General Public License v2.0
57 stars 23 forks source link

End-to-end testing with Cypress #82

Closed cadic closed 2 years ago

cadic commented 2 years ago

Description of the Change

Closes #79

Added end-to-end testing with Cypress. Runs automatically on every pull request to develop branch.

Steps to perform local testing or develop new tests:

  1. Install dependencies npm install
  2. Start wp-env local development suite npm run env:start
  3. Run cypress npm run cypress:run or open developer browser npm run cypress:open

Includes ./tests/bin/set-core-version.js to change WordPress version for wp-env. Usage:

./tests/bin/set-core-version.js WordPress/WordPress#5.2

Testing environment: WordPress latest

Testing suite:

Verification Process

GitHub Actions workflow "End-to-end Testing", expected to pass the test. Refer to "Description of the Change" for local testing.

Checklist:

Changelog Entry

Add - End-to-end testing with Cypress

Credits

Props @cadic, @dinhtungdu, @darylldoyle, @Sidsector9

dinhtungdu commented 2 years ago

@cadic I spent some hours today testing the issue with WordPress#master and other versions than lastest, when we use those versions, there is a file permission issue that prevents the rewrite from working. I installed a plugin to check the content of the .htacess file, it's created only when we testing against latest. The screenshot below is the .htaccess file when I test against WordPress#master, (sorry for the red overlay, but at least we can see the underlying content).

This issue only happens on GitHub Actions, the rewrite works flawlessly testing locally.

This issue was reported on the Gutenberg repo: https://github.com/WordPress/gutenberg/issues/28201

I'm not really sure why the rewrite works on local (mine is a M1 Mac Mini) but not on GitHub Actions, this is something we need to clarify and fix because it will affect our E2E testings for every plugin. When the latest is used, wp-env uses the existing WordPress installation from the Docker image, I guess there are some permission issues downloading and extracting the custom core version. But I'm not sure at this point because it's working locally.

I think, for now, it's good to use only latest version as you're doing because the rewrite issue doesn't belong to our plugin or our Cypress setup.

Admin can login and make sure plugin is activated -- Permalink structure works 16 12 25

cadic commented 2 years ago

@dinhtungdu thank you for looking into this!

Yes, I decided to refuse using other environments rather than latest at this time.