search

microsoft / win-net-test

Networking related test tools for Windows. Relevant for anyone who is interested in Windows networking.
MIT License
25 stars 4 forks source link

CI false signal for functional tests #25

Open nigriMSFT opened 10 months ago

nigriMSFT commented 10 months ago

The CI is showing as passing even though the functional.ps1 is erroring out. This error should cause CI to fail.

D:\a\win-net-test\win-net-test\tools\functional.ps1 : The variable '$RootDir' cannot be retrieved because it has not 
been set.
At D:\a\_temp\5bf[12](https://github.com/microsoft/win-net-test/actions/runs/7649151365/job/20843132494#step:6:13)8[13](https://github.com/microsoft/win-net-test/actions/runs/7649151365/job/20843132494#step:6:14)-891e-4eac-ab63-9df4ef1e02df.ps1:1 char:1
+ tools/functional.ps1 -Verbose -Config Release -Arch x64 -Iterations 1 ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (RootDir:String) [functional.ps1], RuntimeException
    + FullyQualifiedErrorId : VariableIsUndefined,functional.ps1

Example pipeline: https://github.com/microsoft/win-net-test/actions/runs/7649151365/job/20843132494

nigriMSFT commented 10 months ago

FYI as seen with this experimental PR #32 , the issue only happens when the functional.ps1 script errors out due to some syntax error. Most other failures cause the CI pipeline to fail.