Improve unit and e2e test documentation

[tellthemachines]

Is your feature request related to a problem? Please describe.

• When running the unit/integration tests, test results get cached and often fail when attempting to update fixtures.

• There are snapshot tests inside the e2e test suite, but there is no script to update them. They can still be updated manually, but having a script in package.json makes their existence more visible.

• There don't seem to be any docs at all around test/integration/fixtures, what they are used for, and how to update them.

Describe the solution you'd like

• Add docs to packages/e2e-tests/fixtures/blocks/README.md to explain that when updating these fixtures, it may be necessary to run npx jest --clearCache followed by npm run fixtures:regenerate.

• Add test-e2e:update script to root package.json for updating the e2e test snapshots, similar to test-unit:update.

• Add a readme to test/integration/fixtures, or test/integration, or wherever is most visible, explaining how to update these fixtures and what tests they are related to.

Describe alternatives you've considered

Expand info in https://developer.wordpress.org/block-editor/contributors/develop/testing-overview/ - but it would be great to do both :D

[talldan]

Add docs to packages/e2e-tests/fixtures/blocks/README.md to explain that when updating these fixtures, it may be necessary to run npx jest --clearCache followed by npm run fixtures:regenerate.

I made the related issue https://github.com/WordPress/gutenberg/issues/16176. I think this could be considered a bug that we might want to solve. It doesn't seem right that we should have to clear the cache.

Agree on the other points though.

[talldan]

Discovered another thing to add to the list. It seems as though packages/e2e-tests/fixtures/block-transforms.js needs to be updated whenever a new full-content fixture is added.

[noisysocks]

Not totally related to this issue, but something @tellthemachines and I were discussing the other day is how it could be worth looking into improving the developer experience surrounding our E2E tests and E2E tooling.

Some ideas:

• When a test fails, and the environment is Travis, immediately re-run the test.
• When a test fails, save a screenshot page.screenshot() somewhere so that one can see what the page looked like at the time of failure.
• When a test fails, save a representation of the DOM tree somewhere so that one can see what the DOM looked like at the time of failure.
• When a test fails, output a link to troubleshooting documentation that might be helpful.
• Introduce a library of higher level E2E testing functions that wrap Puppeteer functions. For example, a higher level click() function could waitForSelector() (with a short timeout) and then click(). We could even ensure that E2E tests are written using best practices by forbidding the use of lower level Puppeteer functions.

[DaisyOlsen]

Is this still relevant? Is there anything still actionable here?