Make gradual adoption easier

[swissspidy]

If an existing plugin like Yoast SEO or WooCommerce wants to adopt Plugin Check, they are currently being greeted by hundreds of errors and warnings. To make adoption for existing plugins easier, it would be nice if we could provide some sort of baseline of ignored errors. Similar to how PHPStan does it: https://phpstan.org/user-guide/baseline

[mukeshpanchal27]

Thank you, Pascal, for bringing up this issue. I appreciate the idea as it will make it easier for others to adopt.

[ernilambar]

I am testing here https://github.com/ernilambar/plugin-check/pull/9 to implement baseline for Plugin Check.

Baseline file will be in following format:

<?php 
  return [
    'file-1.php' => [ 'code_1', 'code_2' ],
    'subdir/file-2.php' => [ 'code_2' ],
    'another.php' => [ 'code_3' ],
    ...
  ];

Example:

  <?php 
  return [
    'utils.php' => [ 'obfuscated_code_detected', 'WordPress.Security.EscapeOutput.OutputNotEscaped' ],
    'inc/fields/field.php' => [ 'WordPress.Security.EscapeOutput.OutputNotEscaped' ],
    'pc-sample.php' => [ 'WordPress.Security.EscapeOutput.OutputNotEscaped' ],
    'readme.txt' => [ 'trunk_stable_tag', 'stable_tag_mismatch' ],
  ];

Example baseline for Yoast SEO plugin - https://pastebin.com/8ps5ZGPY

--generate-baseline argument in plugin check command will generate baseline file plugin-check-baseline.php and save it in the root of the mentioned plugin.

wp plugin check plugin-slug --generate-baseline

Now in the output of wp plugin check plugin-slug, errors/warnings mentioned in the baseline will be excluded.

Implementation can be found here: https://github.com/ernilambar/plugin-check/pull/9/files

This is just a POC and does not have advanced features. I am expecting reviews and feedbacks. What minimum features should be there in the PR to be accepted?

[swissspidy]

Oh that's very cool!

What minimum features should be there in the PR to be accepted?

That is a great (and tough) question. Right now this is merely an idea. It's not clear whether this is actually useful for plugin developers, so I think we would want to wait for some more feedback once v1 is released. If we decide to go down that route, further testing of your POC would then probably identify missing features quickly if there are any.

After v1 is out we'll probably have to first double down on documentation to properly set expectations and define the various use cases. In most cases it's probably just matter of excluding the PHPCS checks because the project already uses PHPCS separately. For that there's no need for a baseline. But again, more feedback and testing needed.

[swissspidy]

While the POC is very intruiging and I still like the idea, there doesn‘t seem to be enough interest. Closing as maybelater