search

dsccommunity / DscResource.Test

Module to test your Projects against the HQRM guidelines
MIT License
5 stars 8 forks source link

Script Analyzer test should make use of multiple custom rule paths #100

Open johlju opened 3 years ago

johlju commented 3 years ago

The PSSA test source/Tests/QA/PSSAResource.common.Tests.ps1 currently ignores a repository's Script Analyzer file. I Script Analyzer file can be used to add additional custom rules that should only be run for the repository.

Script Analyzer is run with a settings file like this.

Invoke-ScriptAnalyzer -Path .\output\SqlServerDsc\15.0.1\DSCResources\**\*.psm1 -Recurse -Settings .\.vscode\analyzersettings.psd1

But a better option would be to allow custom rule paths to be specified in the build configuration file build.yml under the key DscTest: together with an option to include additional rules (if the rules are not prefixed with Measue-*).

The custom rule paths that is set must point to one or more module or module files. Since Script Analyzer must be called like this:

Invoke-ScriptAnalyzer `
    -Path .\output\SqlServerDsc\15.0.1\DSCResources\DSC_SqlTraceFlag\*.psm1 `
    -CustomRulePath @(
        '.\output\RequiredModules\DscResource.AnalyzerRules'
        '.\tests\QA\AnalyzerRules\SqlServerDsc.AnalyzerRules.psm1'
     ) `
    -IncludeRule @('Measure-*')
gaelcolas commented 3 years ago

I have mixed feelings about this. Adding extra files/rules, ok why not but it should not conflict/replace HQRM tests (that's the goal here, after all). So if someone wants to amend the HQRM tests being run for a pipeline, it should only been done via the build.yaml so that it's clear from a reviewer's perspective.

johlju commented 3 years ago

We could definitely add an array of analyzer rules to the build.yaml. The scenario is that we added repository specific analyzer rules to the SqlServerDsc repo:

https://github.com/dsccommunity/SqlServerDsc/blob/main/tests/QA/AnalyzerRules/SqlServerDsc.AnalyzerRules.psm1

As a workaround we added a test that runs the analyzer rules:

Analyzer rules: https://github.com/dsccommunity/SqlServerDsc/blob/main/tests/QA/ScriptAnalyzer.Tests.ps1

The QA test is run in the pipeline here: https://github.com/dsccommunity/SqlServerDsc/blob/49f78a8b12c0f540ef76cdcbf6605fffa30c17fa/azure-pipelines.yml#L65-L70

It would have been cleaner that these were run during the regular HQRM tests.