search

chef / knife-windows

Plugin for Chef's knife tool for working with Windows nodes
Apache License 2.0
152 stars 110 forks source link

Windows bootstrap does not detect existing chef installation #439

Closed spuder closed 5 years ago

spuder commented 7 years ago

Chef client 12.20.21 Windows 2012r2

Problem

If you install chef on your golden image, the install.ps1 script will not detect it, and will re-download chef when bootstrapping.

Additional Information

We use packer to create windows 2012R2 golden images, (based off of the bento boxes and Matt Wrocks images) One of the packer steps installs chef using omnibus

    {
      "inline": [". { iwr -useb https://omnitruck.chef.io/install.ps1 } | iex; install -channel stable -version {{ user `chef_version` }}"],
      "type": "powershell",
      "elevated_user": "vagrant",
      "elevated_password": "vagrant"
    },

The golden image has chef installed

screenshot 2017-10-25 13 42 07

Yet, every machine that gets bootstrapped, reinstalls chef. When spinning up dozens or hundreds of windows vms, this adds a lot of unnecessary delay. The windows install.ps1 script should behave more like the linux install.sh script and intelligently detect existing chef installs. 

Waiting for remote response before bootstrap.192.0.2.0 .
192.0.2.0 Response received.
Remote node responded after 0.01 minutes.
Bootstrapping Chef on 192.0.2.0
192.0.2.0 Rendering "C:\Users\vagrant\AppData\Local\Temp\bootstrap-12124-1508946731.bat" chunk 1
192.0.2.0 Rendering "C:\Users\vagrant\AppData\Local\Temp\bootstrap-12124-1508946731.bat" chunk 2
192.0.2.0 Rendering "C:\Users\vagrant\AppData\Local\Temp\bootstrap-12124-1508946731.bat" chunk 3
192.0.2.0 Rendering "C:\Users\vagrant\AppData\Local\Temp\bootstrap-12124-1508946731.bat" chunk 4
192.0.2.0 Rendering "C:\Users\vagrant\AppData\Local\Temp\bootstrap-12124-1508946731.bat" chunk 5
192.0.2.0 Rendering "C:\Users\vagrant\AppData\Local\Temp\bootstrap-12124-1508946731.bat" chunk 6
192.0.2.0 Rendering "C:\Users\vagrant\AppData\Local\Temp\bootstrap-12124-1508946731.bat" chunk 7
192.0.2.0 Rendering "C:\Users\vagrant\AppData\Local\Temp\bootstrap-12124-1508946731.bat" chunk 8
192.0.2.0 Checking for existing directory "C:\chef"...
192.0.2.0 Existing directory found, skipping creation.
192.0.2.0
192.0.2.0 C:\Users\vagrant>(
192.0.2.0 echo.url = WScript.Arguments.Named("url")
192.0.2.0  echo.path = WScript.Arguments.Named("path")
192.0.2.0  echo.proxy = null
192.0.2.0  echo.'* Vaguely attempt to handle file:// scheme urls by url unescaping and switching all
192.0.2.0  echo.'* / into .  Also assume that file:/// is a local absolute path and that file://<foo>
192.0.2.0  echo.'* is possibly a network file path.
192.0.2.0  echo.If InStr(url, "file://") = 1 Then
192.0.2.0  echo.url = Unescape(url)
192.0.2.0  echo.If InStr(url, "file:///") = 1 Then
192.0.2.0  echo.sourcePath = Mid(url, Len("file:///") + 1)
192.0.2.0  echo.Else
192.0.2.0  echo.sourcePath = Mid(url, Len("file:") + 1)
192.0.2.0  echo.End If
192.0.2.0  echo.sourcePath = Replace(sourcePath, "/", "\")
192.0.2.0  echo.
192.0.2.0  echo.Set objFSO = CreateObject("Scripting.FileSystemObject")
192.0.2.0  echo.If objFSO.Fileexists(path) Then objFSO.DeleteFile path
192.0.2.0  echo.objFSO.CopyFile sourcePath, path, true
192.0.2.0  echo.Set objFSO = Nothing
192.0.2.0  echo.
192.0.2.0  echo.Else
192.0.2.0  echo.Set objXMLHTTP = CreateObject("MSXML2.ServerXMLHTTP")
192.0.2.0  echo.Set wshShell = CreateObject( "WScript.Shell" )
192.0.2.0  echo.Set objUserVariables = wshShell.Environment("USER")
192.0.2.0  echo.
192.0.2.0  echo.rem http proxy is optional
192.0.2.0  echo.rem attempt to read from HTTP_PROXY env var first
192.0.2.0  echo.On Error Resume Next
192.0.2.0  echo.
192.0.2.0  echo.If NOT (objUserVariables("HTTP_PROXY") = "") Then
192.0.2.0  echo.proxy = objUserVariables("HTTP_PROXY")
192.0.2.0  echo.
192.0.2.0  echo.rem fall back to named arg
192.0.2.0  echo.ElseIf NOT (WScript.Arguments.Named("proxy") = "") Then
192.0.2.0  echo.proxy = WScript.Arguments.Named("proxy")
192.0.2.0  echo.End If
192.0.2.0  echo.
192.0.2.0  echo.If NOT isNull(proxy) Then
192.0.2.0  echo.rem setProxy method is only available on ServerXMLHTTP 6.0+
192.0.2.0  echo.Set objXMLHTTP = CreateObject("MSXML2.ServerXMLHTTP.6.0")
192.0.2.0  echo.objXMLHTTP.setProxy 2, proxy
192.0.2.0  echo.End If
192.0.2.0  echo.
192.0.2.0  echo.On Error Goto 0
192.0.2.0  echo.
192.0.2.0  echo.objXMLHTTP.open "GET", url, false
192.0.2.0  echo.objXMLHTTP.send()
192.0.2.0  echo.If objXMLHTTP.Status = 200 Then
192.0.2.0  echo.Set objADOStream = CreateObject("ADODB.Stream")
192.0.2.0  echo.objADOStream.Open
192.0.2.0  echo.objADOStream.Type = 1
192.0.2.0  echo.objADOStream.Write objXMLHTTP.ResponseBody
192.0.2.0  echo.objADOStream.Position = 0
192.0.2.0  echo.Set objFSO = Createobject("Scripting.FileSystemObject")
192.0.2.0  echo.If objFSO.Fileexists(path) Then objFSO.DeleteFile path
192.0.2.0  echo.Set objFSO = Nothing
192.0.2.0  echo.objADOStream.SaveToFile path
192.0.2.0  echo.objADOStream.Close
192.0.2.0  echo.Set objADOStream = Nothing
192.0.2.0  echo.End If
192.0.2.0  echo.Set objXMLHTTP = Nothing
192.0.2.0  echo.End If
192.0.2.0 ) 1>C:\chef\wget.vbs
192.0.2.0
192.0.2.0 C:\Users\vagrant>(
192.0.2.0 echo.param(
192.0.2.0  echo.   [String] $remoteUrl,
192.0.2.0  echo.   [String] $localPath
192.0.2.0  echo.)
192.0.2.0  echo.
192.0.2.0  echo.$ProxyUrl = $env:http_proxy;
192.0.2.0  echo.$webClient = new-object System.Net.WebClient;
192.0.2.0  echo.
192.0.2.0  echo.if ($ProxyUrl -ne '') {
192.0.2.0  echo.  $WebProxy = New-Object System.Net.WebProxy($ProxyUrl,$true)
192.0.2.0  echo.  $WebClient.Proxy = $WebProxy
192.0.2.0  echo.}
192.0.2.0  echo.
192.0.2.0  echo.$webClient.DownloadFile($remoteUrl, $localPath);
192.0.2.0 ) 1>C:\chef\wget.ps1
192.0.2.0
192.0.2.0 C:\Users\vagrant>(
192.0.2.0
192.0.2.0
192.0.2.0
192.0.2.0 )
192.0.2.0 Detected Windows Version 6.3 Build 9600
192.0.2.0
192.0.2.0 C:\Users\vagrant>goto Version6.3
192.0.2.0
192.0.2.0 C:\Users\vagrant>goto Version6.2
192.0.2.0
192.0.2.0 C:\Users\vagrant>goto architecture_select
192.0.2.0
192.0.2.0 C:\Users\vagrant>IF "AMD64" == "x86" IF not defined PROCESSOR_ARCHITEW6432
192.0.2.0
192.0.2.0 C:\Users\vagrant>goto install
192.0.2.0 Checking for existing downloaded package at "C:\Users\vagrant\AppData\Local\Temp\chef-client-latest.msi"
192.0.2.0 No existing downloaded packages to delete.
192.0.2.0 Attempting to download client package using PowerShell if available...
192.0.2.0 powershell.exe -ExecutionPolicy Unrestricted -InputFormat None -NoProfile -NonInteractive -File  C:\chef\wget.ps1 "https://w.chef.io/chef/download?p=windows&pv=2012&m=x86_64&DownloadContext=PowerShell&v=12" "C:\Users\vagrant\AppData\Local\Temp\chef-client-latest.i"
192.0.2.0 Download via PowerShell succeeded.
192.0.2.0 Installing downloaded client package...
192.0.2.0
192.0.2.0 C:\Users\vagrant>msiexec /qn /log "C:\Users\vagrant\AppData\Local\Temp\chef-client-msi31600.log" /i "C:\Users\vagrant\AppDatLocal\Temp\chef-client-latest.msi"
192.0.2.0 Successfully installed Chef Client package.
192.0.2.0 Installation completed successfully
192.0.2.0 Writing validation key...
192.0.2.0 Validation key written.
192.0.2.0
192.0.2.0 C:\Users\vagrant>mkdir C:\chef\trusted_certs
192.0.2.0

Here is the intelligence in the linux script

https://github.com/chef/chef/blob/db57131ad383076391b9df32d5e9989cfb312d58/lib/chef/knife/bootstrap/templates/chef-full.erb#L165-L176

It runs

  if test -f /usr/bin/chef-client; then
    echo "-----> Existing Chef installation detected"
  else
    echo "-----> Installing Chef Omnibus (<%= latest_current_chef_version_string %>)"
    do_download ${install_sh} $tmp_dir/install.sh
    sh $tmp_dir/install.sh -P chef <%= latest_current_chef_version_string %>
  fi

Here is the windows install.ps1 script

https://github.com/chef/knife-windows/blob/master/lib/chef/knife/bootstrap/windows-chef-client-msi.erb

The two scripts would be more similar if the install.ps1 did something like: 

# Get-Command is available all the way back to powershell 2.0
# https://ss64.com/ps/get-command.html
if (Get-Command 'chef-client' -ErrorAction SilentlyContinue)
{
    Write-Host "-----> Existing Chef installation detected"
}
else {
 # Download and install chef
}

Or a more batch friendly command

chef-client --version 2> NUL
IF NOT %ERRORLEVEL%==9009 ECHO "Existing Chef installation detected"
spuder commented 6 years ago

Different approaches for detecting if chef-client is installed using batch

  1. Try and run command ‘Chef-client.bat —version’
  2. Check path for chef-client.bat file
  3. Where.exe

https://stackoverflow.com/questions/4781772/how-to-test-if-an-executable-exists-in-the-path-from-a-windows-batch-file

Trying to run chef-client.bat —version would be compatible even with really old shells, however it also adds several seconds to the bootstrap process.

RobbertJanSW commented 6 years ago

You could try using my PR #435. It will still download The client, but will continue straight after because I replaced the 5 minute wait for a blocking, non-parallel MSI run.

Also skipping the download won’t be difficult to add to that PR but that would make it a bit messy. It would help if someone merged it.

spuder commented 6 years ago

Brain dump for myself:

Trying to use WHERE since it is native to batch

  @echo Checking for existing chef installation
  @set chef_installed=WHERE chef-client
  @echo !chef_installed!
  @call !chef_installed!
  @set CHEF_CLIENT_ERROR=!ERRORLEVEL!
  @if ERRORLEVEL 0 (
      @echo "Detected existing chef install, skipping download"
  ) else (
10.254.130.165 Detected Windows Version 10.0 Build 14393
10.254.130.165 Warning: Unknown version of Windows, assuming default of Windows 2008r2
10.254.130.165
10.254.130.165 C:\Users\vagrant>goto architecture_select
10.254.130.165
10.254.130.165 C:\Users\vagrant>IF "AMD64" == "x86" IF not defined PROCESSOR_ARCHITEW6432
10.254.130.165
10.254.130.165 C:\Users\vagrant>goto install
10.254.130.165 Checking for existing downloaded package at "C:\Users\vagrant\AppData\Local\Temp\chef-client-latest.msi"
10.254.130.165 No existing downloaded packages to delete.
10.254.130.165 Checking for existing chef installation
10.254.130.165 WHERE chef-client
10.254.130.165 C:\opscode\chef\bin\chef-client
10.254.130.165 ( was unexpected at this time.
10.254.130.165
10.254.130.165 C:\opscode\chef\bin\chef-client.bat
10.254.130.165
10.254.130.165 C:\Users\vagrant>@if NOT ==0 (

There is some issue with delayed evaluation that I have yet to figure out.

Also trying to not save the WHERE command to a variable and execute it directly

  @echo Checking for existing chef installation
  WHERE chef-client
  @set CHEF_CLIENT_STATUS=!ERRORLEVEL!
  @if !CHEF_CLIENT_STATUS!==0 (
      @echo "Detected existing chef install, skipping download"
  ) else (
      @echo Attempting to download client package using PowerShell if available...

Trying to make a new 'function' ( I think that is what batch calls them)

<% else %>
  @set MACHINE_ARCH=x86_64
  IF "%PROCESSOR_ARCHITECTURE%"=="x86" IF not defined PROCESSOR_ARCHITEW6432 @set MACHINE_ARCH=i686
<% end %>
goto chef_installed

:chef_installed
@echo Checking for existing chef installation
WHERE chef-client
If !ERRORLEVEL!==0 (
@echo "Detected existing chef install, skipping download"
) else (
goto install
)

no longer errors, but somehow still enters the :install code

C:\Users\vagrant>IF "AMD64" == "x86" IF not defined PROCESSOR_ARCHITEW6432
10.254.130.113
10.254.130.113 C:\Users\vagrant>goto chef_installed
10.254.130.113 Checking for existing chef installation
10.254.130.113
10.254.130.113 C:\Users\vagrant>WHERE chef-client
10.254.130.113 C:\opscode\chef\bin\chef-client
10.254.130.113
10.254.130.113 C:\opscode\chef\bin\chef-client.bat
10.254.130.113
10.254.130.113 C:\Users\vagrant>If !ERRORLEVEL! == 0 ()  else (goto install )
10.254.130.113 "Detected existing chef install, skipping download"
10.254.130.113 Checking for existing downloaded package at "C:\Users\vagrant\AppData\Local\Temp\chef-client-latest.msi"
10.254.130.113 No existing downloaded packages to delete.
10.254.130.113 Attempting to download client package using PowerShell if available...
10.254.130.113 powershell.exe -ExecutionPolicy Unrestricted -InputFormat None -NoProfile -NonInteractive -File  C:\chef\wget.ps1 "https://www.chef.io/chef/download?p=windows&pv=2008r2&m=x86_64&DownloadContext=PowerShell&v=12" "C:\Users\vagrant\AppData\Local\Temp\chef-client-latest.msi"
10.254.130.113 Download via PowerShell succeeded.
10.254.130.113 Installing downloaded client package...