Introduce list of all sniffers with code examples

[simivar]

One thing that's super useful for new users is code examples. Right now, the biggest problem for someone to start using PHP_CodeSniffer is lack of a list of available Sniffers with their options and examples. From what I know to get a list of all sniffs right now, you have to go through source code in src/Standards, and it's exhausting. Something like mlocati/phpPHP-cs-fixer-configurator could be super useful for new and old users of PHP_CodeSniffer.

From what I see there are .xml files in /docs directory. I couldn't find a place where they are made public, but we could use those to make that list I am talking about.

If there is "service" like that, I am sorry. I've tried to find it in Google, but nothing of the sort came up.

[jrfnl]

I have a feeling you are looking for: phpcs --standard=PSR12 --generator=Text. Alternative generators available: HTML and Markdown.

Doesn't show the configuration options, but does show the sniffs and code examples (if the xml docs are available).

[simivar]

@jrfnl thanks, that's helpful!

What I had in mind is something more interactive, more comfortable to browse (user-friendly) and not old-school. Something like documentation with a list of all available sniffs with their description, code examples, and options. I think it would be helpful for new users, example. Thanks to that entry barrier would be lower.

To achieve that we would have to:

1. Add configuration options to .xml files in /docs.
2. Require .xml file in /docs for every sniff.

Also, we could try to automate it in some way and generate /docs from sniff class. That would require some standardized file-level doc comment with description and examples, but it would be easier to maintain as sniff options would be generated directly from code (there wouldn't be a possibility for options to be out-of-date with sniffer).

Later, it could also have a list of community-developed sniffers so people could easier find those they need as well.

What do you think?

[MasterOdin]

This somewhat reminds me of an issue I made a while back (#2489) where I think the current XML docs are lacking in certain parts of metadata for automatic generation into a /docs structure, and I think @gsherwood's response (https://github.com/squizlabs/PHP_CodeSniffer/issues/2489#issuecomment-491682505) there is relevant here:

Thanks for taking a look at this. I've put any documentation changes on hold as there is a concurrent project about product documentation ongoing where I work and I will probably use the format selected to continue PHPCS docs. This will include how the docs are written and where they are hosted. They will definitely not be XML, so I suspect the entire XML format will disappear in favour of an easier syntax to work with (I don't know what it is yet).

In terms of other projects, I find eslint to be a good gold standard of what a linter documentation can look like (easily searchable, each page easy to understand, code examples of good vs bad, etc.). Far less exhausting that dealing with the docs of phpcs, and also as a nice side-effect, easily reachable/searchable through google.

I would be happy to help / contribute to the set-up of an automated /docs from base XML (or other format) files in that vein of eslint.

[simivar]

@gsherwood any news on the documentation format and how you would like it to be done?

[gsherwood]

any news on the documentation format and how you would like it to be done?

I'd like it done outside the product. Please don't spend time trying to improve the built-in XML-based docs. Not only does nobody use them, but they are a terrible medium for relaying this sort of information.

[simivar]

@gsherwood There still has to be some way to put information about the sniffer in the project and we have to agree on one simple way to do that. Using ReflectionClass we can get information about available options (and their description/type from their docBlock). But from where we should take sniffer description and code examples?

[gsherwood]

But from where we should take sniffer description and code examples?

They would be hand-crafted, as they are here: https://github.com/squizlabs/PHP_CodeSniffer/wiki/Customisable-Sniff-Properties

I don't think end-user documentation should be generated from code comments.

[simivar]

I, myself, was thinking about using as much automation from code as we can, so documentation stays up-to-date. Also, thanks to that source code would be better documented as there would be docblock in every class and on every property describing usage.

To gather proper documentation we need:

1. sniff rule (e.g. Generic.Files.LineLength)
2. sniff description (e.g. It is recommended to keep lines at approximately 80 characters long for better code readability.)
3. sniff options (e.g. lineLimit, int, defaults to 80)
4. sniff code examples

1-3 we can easily grab from code with \ReflectionClass. For 4th one, I was thinking about using tests. That seems like the place where code examples are in source code, but... from what I see tests here are not standard PHPUnit tests and in that form are not easy to use as such.

What do you think about that? Or it still feels weird for you, and you don't want to go that way?

[gsherwood]

Or it still feels weird for you, and you don't want to go that way?

You are obviously free to take the source and produce whatever documentation you want from it. But automated documentation from the source is not something I would accept into the PHPCS core as I don't want to manage that because I don't want to write documentation like that.

I'm grateful that you're trying to help, but the reality is that whatever ends up being produced is something I need to personally manage from that point onwards. So I'm not looking for a quick fix to pump out some level of docs. I'd rather have great documentation for 20% of the sniffs than bad documentation for all of them.

[simivar]

Sure, I get that. That is why I am trying to find some solution where source code gets better, you have less work, and everyone can contribute.

That is why I felt that adding a description of sniffer and option to docblocks would be beneficial for source code (other developers would know from beginning what the class/property is about), and I could do some auto-generated docs outside of this project with that data.

Maybe the solution would be going laminas-like way? Adding a simple docs directory where .md files would sit, what do you think? Thanks to that, everyone could open PRs to improve or add documentation as right now the wiki is limited only to members.

[jrfnl]

@simivar I've been thinking about creating an outside tool - more specifically aimed at external PHPCS standards - to generate sniff-lists, including property information for a while. Output would be in various formats (HTML, markdown). My intention is to add that to PHPCSDevTools, the idea being that sniffs developers can use it to generate docs to put on their website, in a wiki or in a readme and such. If you're interested in contributing to this or joining the discussion on that, please feel invited to open an issue in that repo. I'll try and spend some time writing up my current thoughts on the subject in the near future.

Regarding:

Require .xml file in /docs for every sniff.

PHPCSDevTools already contains a checker tool for that. See PR #2872 for an implementation of its use.

Regarding:

Not only does nobody use them...

I don't agree. People do use them, though it is a little known feature and the "little known" part is the problem more than anything. I mean: if you look at the wiki, aside from the --help block in "Usage", they're not even mentioned anywhere.

... they are a terrible medium for relaying this sort of information.

More than anything because the current Generator setup doesn't allow for all the information which should be communicated in end-user documentation.

[Abdillah]

This is a great references: https://eslint.org/docs/rules/. In which each properties is described with the effect it caused and if available the code example.