Running tests via Pester and PowerShell ISE causes failures

[AwaNoodle]

Running the tests using Invoke-Pester rather than using the route via the OneBuild.bat gives me two failures, one for getting the latest nuget.exe path and one for the latest nunit path.

I am loading in the Pester module that is used by OneBuild:

> Import-Module C:\Sandbox\OneBuild\packages\pester.3.0.0-beta2\tools\Pester.psm1

From the OneBuild root, I am then running:

PS C:\Sandbox\OneBuild> Invoke-Pester .\tests

This gives me the two failures:

[-] Should return the full path to to the highest version of NuGet.Commandline found in the solution packages folder 346ms
      Expected: {C:\Users\Administrator\AppData\Local\Temp\1\83bfad2e-f926-44c2-bd4f-19a856c7458d\packages\NuGet.CommandLine.2.7.3\tools\nuget.exe}, But was {C:\Users\Administrator\AppData\Local\Temp\1\83bfad
2e-f926-44c2-bd4f-19a856c7458d\packages\NuGet.CommandLine.2.7.2\tools\nuget.exe}
      at line: 129 in C:\Sandbox\OneBuild\tests\CommonFunctions.Tests.ps1

and

[-] Should return the full path to to the highest version of NUnit.Runners found in the solution packages folder 313ms
      Expected: {C:\Users\Administrator\AppData\Local\Temp\1\a81b2ff3-d9e9-47c0-9770-f341c2c5982e\packages\NUnit.Runners.2.6.3\tools\nunit-console.exe}, But was {C:\Users\Administrator\AppData\Local\Temp\1\a8
1b2ff3-d9e9-47c0-9770-f341c2c5982e\packages\NUnit.Runners.2.6.2\tools\nunit-console.exe}
      at line: 161 in C:\Sandbox\OneBuild\tests\CommonFunctions.Tests.ps1

I don't see these if I run via the OneBuild.bat

[AwaNoodle]

Forgot to add: I modified the test to show a tree output just before it runs Get-NuGetPath. I got this result:

C:\USERS\ADMINISTRATOR\APPDATA\LOCAL\TEMP\1\83BFAD2E-F926-44C2-BD4F-19A856C7458D
ÀÄÄÄpackages
    ÃÄÄÄNuGet.CommandLine.2.7.2
    ³   ÀÄÄÄtools
    ÀÄÄÄNuGet.CommandLine.2.7.3
        ÀÄÄÄtool

[lholman]

@AwaNoodle Hmmm, that's odd, that's a failure of this test..

https://github.com/lholman/OneBuild/blob/master/tests/CommonFunctions.Tests.ps1#L103

and this one..

https://github.com/lholman/OneBuild/blob/master/tests/CommonFunctions.Tests.ps1#L135

Both these tests use the Pester $TestDrive to add multiple versions of the associated .exe files to prove we always retrieve the latest if there are multiple versions restored in the OneBuild\Packages folder, it would seem they are returning the lower, rather than the higher versions.

1. Do you have another version of Pester in your path, can you copy the full output here of running the following? PS C:\Sandbox\OneBuild> Invoke-Pester .\tests -TestName "CommonFunctions.Get-NUnitPath"
2. Does running the following first PS C:\Sandbox\OneBuild> OneBuild.bat -task Invoke-OneBuildUnitTests result in passed tests if you then run run Pester directly as above?

Other thoughts

• I've seen some oddities with the Pester $TestDrive with tests in the same Describe block, but these are isolated in different Describe blocks so that shouldn't be the case here.
• What version of Windows and Powershell are you using?
• It would seem the -Descending parameter in the Sort-Object isn't being adhered to on your system https://github.com/lholman/OneBuild/blob/master/tools/powershell/modules/CommonFunctions.psm1#L39

[AwaNoodle]

1. No other Pester on the machine
2. Same result

I'm running Windows 2k8 with Powershell 4. I am running it through the ISE (64bit).

When I write out the exact same command by hand and create the files in the same structure by hand, I get the result I expect.

[lholman]

1. What do you mean by "When I write out the exact same command by hand and create the files in the same structure by hand, I get the result I expect."?
2. Could you try running this in a Powershell Command Prompt instead of the ISE?

[lholman]

@AwaNoodle Could you copy the output in here as I asked above please?

Do you have another version of Pester in your path, can you copy the full output here of running the following? PS C:\Sandbox\OneBuild> Invoke-Pester .\tests -TestName "CommonFunctions.Get-NUnitPath"

[AwaNoodle]

1. I write out the commands on the PS console to check how it works
2. Works fine in the console.

Output from the test:

PS C:\Sandbox\OneBuild> Invoke-Pester .\tests -TestName "CommonFunctions.Get-NUnitPath"
Executing all tests in 'C:\Sandbox\OneBuild\tests' matching test name 'CommonFunctions.Get-NUnitPath'
Describing CommonFunctions.Get-NUnitPath
   Context When there is more than one version of NuGet.Commandline installed

    Directory: C:\Users\Administrator\AppData\Local\Temp\1\36230cbc-712e-45e1-ab7e-430773622a7d

Mode                LastWriteTime     Length Name                                                                                                                                                              
----                -------------     ------ ----                                                                                                                                                              
d----        10/04/2015     14:10            packages                                                                                                                                                          

    Directory: C:\Users\Administrator\AppData\Local\Temp\1\36230cbc-712e-45e1-ab7e-430773622a7d\packages

Mode                LastWriteTime     Length Name                                                                                                                                                              
----                -------------     ------ ----                                                                                                                                                              
d----        10/04/2015     14:10            NUnit.Runners.2.6.2                                                                                                                                               

    Directory: C:\Users\Administrator\AppData\Local\Temp\1\36230cbc-712e-45e1-ab7e-430773622a7d\packages\NUnit.Runners.2.6.2

Mode                LastWriteTime     Length Name                                                                                                                                                              
----                -------------     ------ ----                                                                                                                                                              
d----        10/04/2015     14:10            tools                                                                                                                                                             

    Directory: C:\Users\Administrator\AppData\Local\Temp\1\36230cbc-712e-45e1-ab7e-430773622a7d\packages\NUnit.Runners.2.6.2\tools

Mode                LastWriteTime     Length Name                                                                                                                                                              
----                -------------     ------ ----                                                                                                                                                              
-a---        10/04/2015     14:10          0 nunit-console.exe                                                                                                                                                 

    Directory: C:\Users\Administrator\AppData\Local\Temp\1\36230cbc-712e-45e1-ab7e-430773622a7d\packages

Mode                LastWriteTime     Length Name                                                                                                                                                              
----                -------------     ------ ----                                                                                                                                                              
d----        10/04/2015     14:10            NUnit.Runners.2.6.3                                                                                                                                               

    Directory: C:\Users\Administrator\AppData\Local\Temp\1\36230cbc-712e-45e1-ab7e-430773622a7d\packages\NUnit.Runners.2.6.3

Mode                LastWriteTime     Length Name                                                                                                                                                              
----                -------------     ------ ----                                                                                                                                                              
d----        10/04/2015     14:10            tools                                                                                                                                                             

    Directory: C:\Users\Administrator\AppData\Local\Temp\1\36230cbc-712e-45e1-ab7e-430773622a7d\packages\NUnit.Runners.2.6.3\tools

Mode                LastWriteTime     Length Name                                                                                                                                                              
----                -------------     ------ ----                                                                                                                                                              
-a---        10/04/2015     14:10          0 nunit-console.exe                                                                                                                                                 
    [-] Should return the full path to to the highest version of NUnit.Runners found in the solution packages folder 193ms
      Expected: {C:\Users\Administrator\AppData\Local\Temp\1\36230cbc-712e-45e1-ab7e-430773622a7d\packages\NUnit.Runners.2.6.3\tools\nunit-console.exe}, But was {C:\Users\Administrator\AppData\Local\Temp\1\36
230cbc-712e-45e1-ab7e-430773622a7d\packages\NUnit.Runners.2.6.2\tools\nunit-console.exe}
      at line: 161 in C:\Sandbox\OneBuild\tests\CommonFunctions.Tests.ps1
Tests completed in 193ms
Passed: 0 Failed: 1

[lholman]

@AwaNoodle I can reproduce, I now get the same two tests failing using the ISE and running Pester directly too.

[AwaNoodle]

@lholman Did you try 32bit or 64bit? I've not tried 32bit yet.

[lholman]

@AwaNoodle Furthermore, it doesn't fail in a Powershell command prompt, only in the ISE.

[lholman]

Fails in 32 and 64bit ISE from what I can tell, can you expand more on "I write out the commands on the PS console to check how it works", do you mean console or ISE? The problem is the ISE here from what I can tell.

[lholman]

@AwaNoodle Found it...

1. Create the following folder structure under c:\temp, make sure you add nunit-console.exe in to each tools folder
2. Open up Powershell console and run PS C:\temp\OneBuild> Get-ChildItem . -Recurse | Where-Object {$_.Name -like 'nunit-console.exe'} | Where-Object {$_.FullName -like '*nunit.runners*'} | Sort-Object $_.FullName -Descending | Select-Object FullName -First 1 | foreach {$_.FullName}
3. Open up Powershell ISE (32 or 64bit) and run PS C:\temp\OneBuild> Get-ChildItem . -Recurse | Where-Object {$_.Name -like 'nunit-console.exe'} | Where-Object {$_.FullName -like '*nunit.runners*'} | Sort-Object $_.FullName -Descending | Select-Object FullName -First 1 | foreach {$_.FullName}

Observe the differences, it would seem that the ISE doesn't adhere to the combination of -Recurse and Sort-Object -Descending parameters

[AwaNoodle]

I guess it's a non-issue then :) Ok to close?

[lholman]

@AwaNoodle I imagine if we re-write the query it may work in both!?