PHPCSStandards / PHP_CodeSniffer

PHP_CodeSniffer tokenizes PHP files and detects violations of a defined set of coding standards.
BSD 3-Clause "New" or "Revised" License
880 stars 54 forks source link

Generators do not always respect `--sniffs` CLI argument #592

Open jrfnl opened 1 month ago

jrfnl commented 1 month ago

Describe the bug

Okay, so this is a weird one and I haven't been able to create a reliable reproduction case.

// Will show the docs for the one sniff.
$ phpcs --generator=Text --standard=Stnd --sniffs=Stnd.Category.SniffName

// Will sometimes show the docs for the one sniff, but will also sometimes show the docs for the complete standard in use (`phpcs.xml.dist`).
$ phpcs --generator=Text --sniffs=Stnd.Category.SniffName

I've tried to create a reproduction scenario to track down the issue, but I haven't been able to - other than with a new sniff in an external standard which isn't public yet.

// Shows the docs based on the `phpcs.xml.dist` file, including the new sniff (PHPCompatibility itself is included in the `phpcs.xml.dist` ruleset).
$ phpcs --generator=Text --sniffs=PHPCompatibility.ParameterValues.RemovedXmlSetHandlerCallbackUnset

// Shows the docs for the one sniff.
$ phpcs --generator=Text --standard=PHPCompatibility --sniffs=PHPCompatibility.ParameterValues.RemovedXmlSetHandlerCallbackUnset

More than anything, I'm opening this issue as a reminder to investigate and to figure out a reproduction case which reliably triggers the bug.

Expected behavior

// Show the docs for the one sniff.
$ phpcs --generator=Text --standard=Stnd --sniffs=Stnd.Category.SniffName

// Show the docs for the one sniff if the sniff is included in the ruleset PHPCS defaults to
// (`[.]phpcs.xml[.dist]` or PEAR if no custom ruleset found).
$ phpcs --generator=Text --sniffs=Stnd.Category.SniffName

// Show "No sniffs registered" if the sniff is NOT included in the ruleset PHPCS defaults to.
$ phpcs --generator=Text --sniffs=Stnd.Category.SniffName

Versions (please complete the following information)

Operating System not relevant
PHP version not relevant
PHP_CodeSniffer version master
Standard see description
Install type git clone

Additional context

I've seen this issue on and off for years now, but haven't dug in deep to pinpoint the exact trigger as adding the --standard=Stnd CLI arg seems to fix the issue.

If I remember correctly, I've predominantly seen this behaviour when testing docs for external standards. Not for docs for any of the PHPCS native standards. (which is part of the reason why creating a reproduction case is more tricky)

Please confirm