Windows Box: kitchen test and kitchen converge && kitchen verify have different $env:PATH

[ehaselwanter]

Description

I have a cookbook installing java on windows. kitchen test does not pick up the changed $env:PATH in the verify/inspec step. But kitchen converge && kitchen verify works

simplified example code:

script = <<-EOH
. java "com.swiftmq.admin.cli.CLI"
EOH

describe powershell(script) do
  its('stdout') { should match /router1/ }
  its('stderr') { should eq '' }
end

fails cause it does not find java, but the following works

script = <<-EOH
$env:PATH += ";C:\\ProgramData\\Oracle\\Java\\javapath"
. java "com.swiftmq.admin.cli.CLI"
EOH

describe powershell(script) do
  its('stdout') { should match /router1/ }
  its('stderr') { should eq '' }
end

Kitchen Version

kitchen --version
Test Kitchen version 1.19.2

ChefDK Version

If you are running test-kitchen via ChefDK, chef --version will provide additional relevent version details.

chef --version
Chef Development Kit Version: 2.4.17
chef-client version: 13.6.4
delivery version: master (73ebb72a6c42b3d2ff5370c476be800fee7e5427)
berks version: 6.3.1
kitchen version: 1.19.2
inspec version: 1.45.13

Platform Version

macOS 10.13.3

Replication Case

anything which updates $env in converge

Kitchen Output

kitchen converge
kitchen verify

has $env:PATH

  Path                           C:\ProgramData\Oracle\Java\javapath;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\

and works.

kitchen test

has $env:PATH

  Path                           C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\

and fails with

     ∅  should eq ""

     expected: ""
          got: "The term 'java' is not recognized as the name of a cmdlet, function, script file, or operable progra...java:String) [], CommandNotFoundException\n    + FullyQualifiedErrorId : CommandNotFoundException\n"

     (compared using ==)

     Diff:
     @@ -1 +1,7 @@
     +The term 'java' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
     +At line:4 char:3
     ++ . java `
     ++   ~~~~
     +    + CategoryInfo          : ObjectNotFound: (java:String) [], CommandNotFoundException
     +    + FullyQualifiedErrorId : CommandNotFoundException

does not. I guess there is a re-use of the open connection which does not receive the updated env. This is just a wild guess.

Kitchen Diagnose

not an issue I guess (and contains lots of internals I need to scrape first)

[cheeseplus]

Pretty sure this is a side effect of the fact that WinRM isn't an actual shell and so it'll never pick up the path change in the single run. I'll spend some time looking at this but pretty sure this isn't something we can do much about given it's how WinRM and such work.

[ehaselwanter]

what about simulating the converge & verify behaviour. terminate winrm session and create a new one. wonder if this the reason I had to get a login shell in inspec tests on linux as well. that would explain that too

[JCapriotti]

I think I am running into this too, via installing a Chocolatey package (e.g. git) and then using Inspec to verify that git --version returns a status of 0. It works when I run kitchen converge and kitchen verify separately, but not if I only run kitchen verify.

Just curious if @ehaselwanter suggestion about terminating the WinRM session and creating a new one has any traction? Or, if there's any other ideas?

[ehaselwanter]

for now I ended up with adding the 'known' needed env/path values to the test execution. not ideal.