search

DataDog / chef-datadog

Chef cookbook for Datadog Agent & Integrations
https://www.datadoghq.com
Apache License 2.0
97 stars 262 forks source link

datadog cookbook constantly failing on windows servers #536

Open benorgil opened 6 years ago

benorgil commented 6 years ago

Not sure whats causing this, but every few weeks all my windows chef runs start failing because of this datadog cookbook. I have the version pinned to 2.11.0 so it should not be updating the client, but it looks like it does periodically try to reinstall?

This kills me because every time this happens I have to interactively log into each server and run this command for it to force uninstall so the chef run will succeed: msiexec /qn /i "c:\chef\cache\ddagent-cli.msi" /norestart ALLUSERS=1 /log dd-agent-install-log.txt NOTUPGRADE=1

Is it trying to reinstall the msi? why?

Errors from failed runs:

Recipe: datadog::dd-agent

olivielpeau commented 5 years ago

Hi @benorgil and sorry for the very late reply.

To pin the version of the Agent that's installed by the cookbook, you should make sure agent_package_action is left to its default or set to "install", and use the agent6_version (Agent v6) or agent_version (Agent 5) attribute (see documentation in default attribute file: https://github.com/DataDog/chef-datadog/blob/master/attributes/default.rb, please use the one from your version of the cookbook).

Also, please note that, currently, the cookbook will reinstall the Agent if you clear Chef's cache directory. Unfortunately this was done to support older versions of Chef (<12.6), we'll attempt to drop support for these old versions and improve the logic in v3.0 of the present cookbook.

Does this info help? Which version of Chef are you running?

bwillif1 commented 5 years ago

I'm getting the same error code '1603' whenever datadog upgrades the agent. I have datadog pinned to version 6.11.0 in chef, so some of my old clients are upgrading. It removes the agent with no problem but fails to re-install it. I experience the same issue when trying to install the msi manually from the server itself.

Browsing on the datadog site for properly uninstalling the agent https://docs.datadoghq.com/agent/faq/how-do-i-uninstall-the-agent/?tab=agentv6 I see this statement: "It’s important that the original account used to install the Agent is also used to remove it, otherwise it’s possible remnants are left behind and it won’t be cleanly removed." I'm thinking this is what is contributing to the failing agent installs.

However, the user who previously manually installed an older version of datadog left the company and their account no longer exists. Does this mean I can never install datadog on these servers again? There has to be a better way to handle uninstalls/installs.

Swampcritter commented 4 years ago

We are running the datadog cookbook through the use of a in-house wrapper to enable log aggregation. The Linux side of the cookbook works fine, however it keeps failing on the Windows installs.

chef-client -r recipe[wabtec_datadog] +---------------------------------------------+ ✔ 2 product licenses accepted. +---------------------------------------------+ Starting Chef Infra Client, version 15.9.17 resolving cookbooks for run list: ["wabtec_datadog"] Synchronizing Cookbooks:

  • chef_handler (3.0.3)
  • yum (5.1.0)
  • apt (7.3.0)
  • datadog (4.3.0)
  • wabtec_datadog (0.3.7) Installing Cookbook Gems: Compiling Cookbooks... [2020-05-08T09:49:33-04:00] WARN: Resource chef_handler from the client is overriding the resource from a cookbook. Please upgrade your cookbook or remove the cookbook from your run_list. Converging 12 resources Recipe: datadog::dd-agent
  • ruby_block[datadog-api-key-unset] action run (skipped due to only_if) Recipe: datadog::_install-windows
  • windows_package[Datadog Agent removal] action nothing (skipped due to action :nothing)
  • remote_file[C:\chef\cache/ddagent-cli.msi] action create
  • create new file C:\chef\cache/ddagent-cli.msi
  • update content in file C:\chef\cache/ddagent-cli.msi from none to f250d6 (file sizes exceed 10000000 bytes, diff output suppressed)
  • remote_file[C:\chef\cache/fix_6_14.ps1] action nothing (skipped due to action :nothing)
  • powershell_script[datadog_6.14.x_fix] action nothing (skipped due to action :nothing)
  • windows_package[Datadog Agent] action install
================================================================================
Error executing action `install` on resource 'windows_package[Datadog Agent]'
================================================================================

Mixlib::ShellOut::ShellCommandFailed
------------------------------------
Expected process to exit with [0, 3010], but received '1603'
---- Begin output of msiexec /qn /i "c:\chef\cache\ddagent-cli.msi"  /norestart ALLUSERS=1 ----
STDOUT:
STDERR:
---- End output of msiexec /qn /i "c:\chef\cache\ddagent-cli.msi"  /norestart ALLUSERS=1 ----
Ran msiexec /qn /i "c:\chef\cache\ddagent-cli.msi"  /norestart ALLUSERS=1 returned 1603

Resource Declaration:
---------------------
# In C:/chef/cache/cookbooks/datadog/recipes/_install-windows.rb

112: windows_package 'Datadog Agent' do # ~FC009
113:   source temp_file
114:   installer_type resolved_installer_type
115:   options install_options
116:   timeout node['datadog']['windows_msi_timeout']
117:   action :install
118:   # Before 3.0.0, we adviced users to use the windows cookbook ~> 1.38.0,
119:   # we should probably keep the compatibilty for some time.
120:   if respond_to?(:returns)
121:     returns [0, 3010]
122:   else
123:     success_codes [0, 3010]
124:   end
125:   not_if do

Compiled Resource:
------------------
# Declared in C:/chef/cache/cookbooks/datadog/recipes/_install-windows.rb:112:in `from_file'

windows_package("Datadog Agent") do
  package_name "Datadog Agent"
  action [:install]
  default_guard_interpreter :default
  declared_type :windows_package
  cookbook_name "datadog"
  recipe_name "_install-windows"
  timeout 900
  source "c:\\chef\\cache\\ddagent-cli.msi"
  installer_type :msi
  options "/norestart ALLUSERS=1"
  returns [0, 3010]
  version "7.19.1.1"
  not_if { #code block }
end

System Info:
------------
chef_version=15.9.17
platform=windows
platform_version=10.0.14393
ruby=ruby 2.6.6p146 (2020-03-31 revision 67876) [x64-mingw32]
program_name=C:/opscode/chef/bin/chef-client
executable=C:/opscode/chef/bin/chef-client

Running handlers: [2020-05-08T09:53:22-04:00] ERROR: Running exception handlers Running handlers complete [2020-05-08T09:53:22-04:00] ERROR: Exception handlers complete Chef Infra Client failed. 1 resources updated in 03 minutes 58 seconds [2020-05-08T09:53:22-04:00] FATAL: Stacktrace dumped to C:/chef/cache/chef-stacktrace.out [2020-05-08T09:53:22-04:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report [2020-05-08T09:53:22-04:00] FATAL: Mixlib::ShellOut::ShellCommandFailed: windows_package[Datadog Agent] (datadog::_insta ll-windows line 112) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0, 3010], but received '1603' ---- Begin output of msiexec /qn /i "c:\chef\cache\ddagent-cli.msi" /norestart ALLUSERS=1 ---- STDOUT: STDERR: ---- End output of msiexec /qn /i "c:\chef\cache\ddagent-cli.msi" /norestart ALLUSERS=1 ---- Ran msiexec /qn /i "c:\chef\cache\ddagent-cli.msi" /norestart ALLUSERS=1 returned 1603