search

squizlabs / PHP_CodeSniffer

PHP_CodeSniffer tokenizes PHP files and detects violations of a defined set of coding standards.
BSD 3-Clause "New" or "Revised" License
10.66k stars 1.48k forks source link

Reports: add end-to-end tests #3804

Open jrfnl opened 1 year ago

jrfnl commented 1 year ago

... for quite a few report types.

The Reports which PHPCS can generate, were until now not covered by tests and therefore had no safeguards/QA whatsoever.

This commit sets up an initial end-to-end test suite with tests for common report types. It doesn't have an opinion on whether or not the reports as currently generated are formatted correctly. For now, it just codifies the current functionality.

For lack of documentation about this format in the PHPUnit docs, the test docs for PHP Core are the closest available to explain the available sections and how to apply these: https://qa.php.net/phpt_details.php

With this initial setup done, it should be fairly straight-forward to expand this end-to-end test suite to cover more report types and to add tests for other command-line options as well. Mind: these type of tests are generally slow, so unit tests/integration tests should be preferred.

jrfnl commented 1 year ago

Rebased to get past merge conflict.

jrfnl commented 1 year ago

I'm moving this PR back to draft as it looks like these tests may need some adjustments to work with PHPUnit 8/9 and I'd rather give priority to PR #3803, which unlocks the use of PHPUnit 8/9 for PHPCS itself as well as for tests for external standards which are based on the PHPCS test framework.

I will remove the tests from PR #3810, so that PR is no longer blocked by this one.