search

microsoft / DevSkim

DevSkim is a set of IDE plugins, language analyzers, and rules that provide security "linting" capabilities.
MIT License
902 stars 115 forks source link

Add tests to identify rules with missing or incomplete guidance #613

Closed danfiedler-msft closed 4 months ago

danfiedler-msft commented 5 months ago

This PR adds two data-driven tests that verify rules:

  1. Specify a guidance file and that file exists
  2. The guidance file does not contain "TODO" or "TO DO"

It also adds guidance or points rules to the appropriate guidance file such that every default rule now has a guidance file.

The second test is temporarily marked as [Ignore] to allow builds to pass. I plan to work on updating the incomplete (those with TODOs) in the near future which will allow us to remove [Ignore] from that test.

gfs commented 5 months ago

Converted to draft so its slightly clearer not to merge yet (though the failing PR tests also give a hint.

gfs commented 5 months ago

Implementation of the tests looks spot on to me. Should sync on what to do about fixing the gaps/if they should be fatal yet as you mentioned.