MethodsAndPractices / vsteam

PowerShell module for accessing Azure DevOps Services and Azure DevOps Server (formerly VSTS or TFS)
https://methodsandpractices.github.io/vsteam-docs/
MIT License
442 stars 155 forks source link

Runing tests on local fail some test, but running from Github actions not #450

Closed mnieto closed 2 years ago

mnieto commented 2 years ago

Steps to reproduce

Running last version (7.5.0) tests just after a git clone, fail 3 tests

.\Build-Module.ps1 -runTests -testOutputLevel Detailed

Output:

Tests Passed: 793, Failed: 9, Skipped: 0 NotRun: 0
BeforeAll \ AfterAll failed: 3
  - VSTeamWiki
  - VSTeamWiki
  - VSTeamWorkItemTag

The error for all 3 tests is very similar

Running tests from 'C:\Users\Miguel\Documents\progs\vsteamLast\Tests\function\tests\Remove-VSTeamWiki.Tests.ps1'
[-] Describe VSTeamWiki failed
  CommandNotFoundException: Could not find Command Get-VSTeamProject

But reproducing the same exact steps as defined in the GitHub actions

       Import-Module ./dist/*.psd1
        Import-Module Pester
        $pesterArgs = [PesterConfiguration]::Default
        $pesterArgs.Run.Exit = $true
        $pesterArgs.Run.Throw = $true
        $pesterArgs.Run.PassThru = $false
        $pesterArgs.TestResult.Enabled = $true
        $pesterArgs.Output.Verbosity = 'Normal'
        $pesterArgs.Run.Path = './Tests/function'
        $pesterArgs.TestResult.OutputFormat = 'JUnitXml'
        $pesterArgs.TestResult.OutputPath = 'test-results.xml'
        Invoke-Pester -Configuration $pesterArgs

There is no error

Tests completed in 76.22s
Tests Passed: 802, Failed: 0, Skipped: 0 NotRun: 0

Expected behavior

Both ways of running test should be coherent. So runing tests on local while modifying code do not throw false positive errors.

Actual behavior?

Running tests from \Build-Module.ps1 script throws errors on tests that passed in the CI server

On Which OS have you tried it?

Windows

What was your server version?

Azure DevOps Services

Other server version

No response

Log output of used API

N/A. Tests ran from source code

Log output of $PSVersionTable

Name                           Value
----                           -----
PSVersion                      7.2.1
PSEdition                      Core
GitCommitId                    7.2.1
OS                             Microsoft Windows 10.0.19043
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0
SebastianSchuetze commented 2 years ago

Interesting. Got the same problem.

SebastianSchuetze commented 2 years ago

@mnieto can you checkout branch from PR #451 and confirm it works now?

mnieto commented 2 years ago

Hi, thanks @SebastianSchuetze for the quick response. Yes, now is working fine

SebastianSchuetze commented 2 years ago

gonna merge this now