Document Windows service using NSSM

[armon]

Just came across this, seems suitable to run Consul as a service. http://nssm.cc

[ryanbreen]

We have some experience using WSW + Chocolatey on our Windows clients. I can provide some detail on that.

On Wed, Mar 4, 2015 at 6:09 PM, Armon Dadgar notifications@github.com wrote:

Just came across this, seems suitable to run Consul as a service. http://nssm.cc

— Reply to this email directly or view it on GitHub https://github.com/hashicorp/consul/issues/758.

[ryanbreen]

That is, winsw: https://github.com/kohsuke/winsw

[armon]

That would be great, I do get this question a lot. We should consider maybe a "Using Windows" guide for some of the oddities on that platform

[highlyunavailable]

I like NSSM, I use it to run Consul and Heka on production Windows servers. A "Using Windows" guide would be much appreciated, I'm running mine in exactly the same config as Linux and it seems ok, but I'm not sure what to look for performancewise.

As a reference, here's my Salt states for what I'm doing for NSSM (with a lot of snipping):

{% if grains['num_cpus'] == 1 %}
{% set GOMAXPROCS = 2 %}
{% else %}
{% set GOMAXPROCS = grains['num_cpus'] %}
{%- endif -%}

Ensure Consul service is registered:
  cmd.run:
    - name: |
        C:\opt\nssm.exe install consul "C:\opt\consul\consul.exe" agent -config-dir "c:\opt\consul\consul.d"
    - unless: sc query consul

Ensure GOMAXPROCS is set to the number of CPUs:
  cmd.run:
    - shell: powershell
    - name: |
        C:\opt\nssm.exe set consul AppEnvironmentExtra GOMAXPROCS={{ GOMAXPROCS }}
    - unless: |
        if ($(C:\opt\nssm.exe get consul AppEnvironmentExtra).Contains("GOMAXPROCS={{ GOMAXPROCS }}")) {exit 0}; exit 1

Ensure negative DNS resolutions are not cached:
  cmd.run:
    - name: |
        C:\Windows\System32\reg.exe add HKLM\SYSTEM\CurrentControlSet\Services\Dnscache\Parameters /v MaxNegativeCacheTtl /t REG_DWORD /d 0 /f
    - unless: |
        C:\Windows\System32\reg.exe query HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\DNSCache\Parameters /v MaxNegativeCacheTtl | findstr /C:0x0

Ensure DNS Client is restarted:
  module.wait:
    - name: service.restart
    - m_name: Dnscache
    - onchanges:
      - cmd: Ensure negative DNS resolutions are not cached

Ensure Consul firewall TCP port is open:
  win_firewall.add_rule:
    - name: Consul Serf LAN
    - localport: 8301

Ensure Consul firewall UDP port is open:
  win_firewall.add_rule:
    - name: Consul Serf LAN UDP
    - localport: 8301
    - protocol: udp

Ensure Consul service is started:
  service.running:
    - name: consul
    - enable: true
{% if grains['os'] != 'Windows' %}
    - reload: true
{% endif %}

[ryanbreen]

That is a seriously awesome config!

[highlyunavailable]

Oh and I didn't say it but I'd be happy to help with the docs as well.

[ryanbreen]

@highlyunavailable Please go for it: your NSSM setup is cooler than our WinSW config. :smile:

[StefanScherer]

@armon Just want to let you know that we are working to build a MSI package with NSSM and consul.exe running as an automatic build at AppVeyor. I'm sure I can get this open sourced very soon.

[megakid]

Not quite as nice as @highlyunavailable but we've got a similar Deploy.ps1 script for our Octopus Deploy rollout:

$release_dir = $OctopusParameters['OctopusOriginalPackageDirectoryPath']
$configFile = $OctopusParameters['Consul.ConfigFile']

$serviceName = "Consul"
$exe = "consul.exe"

try 
{
    Write-Output "Stopping NSSM $serviceName service..."
    Start-Process -Wait "nssm" -ArgumentList "stop $serviceName"
    Write-Output "Removing NSSM $serviceName service..."
    Start-Process -Wait "nssm" -ArgumentList "remove $serviceName confirm"
} 
catch 
{
}

Write-Output "Installing NSSM for $serviceName service with $configFile in $release_dir"
Start-Process -Wait "nssm" -ArgumentList "install $serviceName $exe agent -config-file=$configFile"
Start-Process -Wait "nssm" -ArgumentList "set $serviceName AppDirectory $release_dir"
Start-Process -Wait "nssm" -ArgumentList "set $serviceName DisplayName Consul Daemon"
Start-Process -Wait "nssm" -ArgumentList "set $serviceName Description Consul Daemon that provides distributed KVP and service discovery"
# Allow 15 seconds for graceful shutdown before process is terminated
Start-Process -Wait "nssm" -ArgumentList "set $serviceName AppStopMethodConsole 15000"

# Pipe output to daemon.log
Write-Output "Redirecting StdErr for $serviceName to $release_dir\log\stderr.log"
Start-Process -Wait "nssm" -ArgumentList "set $serviceName AppStderr $release_dir\log\stderr.log"
Write-Output "Redirecting StdOut for $serviceName to $release_dir\log\stdout.log"
Start-Process -Wait "nssm" -ArgumentList "set $serviceName AppStdout $release_dir\log\stdout.log"

Write-Output "Configuring recovery options for $serviceName service..."
Start-Process -Wait "sc" -ArgumentList "failure $serviceName reset= 120 actions= restart/60000/restart/60000//60000"

Write-Output "Starting $serviceName service..."
Start-Process -Wait "nssm" -ArgumentList "start $serviceName"

We use a separate deployment step to set the environment var GOMAXPROCS as until I saw this issue, I was unaware we can set it on NSSM on a per service level...

[carlpett]

I've recently created a WiX/.msi installer for Consul that we use internally (before I found this thread). Would there be interest for adding it upstream? It uses winsw rather than nssm, though, since nssm warns about having multiple instances of it, and saltstack bundles it already.

[StefanScherer]

In the meantime we also have created a WiX MSI package for Consul using nssm.exe. We don't have problem with multiple nssm.exe as we install it per microservice for a dozen of MSI packages running on one machine. I think it would be good idea to share this here or in different PR's so everbody can review it and we get the best out of all :-)

[vLifeOfBrian]

@carlpett @StefanScherer I would be interested in seeing both these packages. I'm in the process of configuring consul agent on a Windows server.

[carlpett]

@vLifeOfBrian: I will do some minor cleanup to get rid of some specifics, then get a PR up later today.

[StefanScherer]

@vLifeOfBrian @carlpett I was allowed to open source parts of our MSI build process. I've put it into an extra repo that can build the MSI package and deploy it as GitHub release as an example. All done at AppVeyor CI. Have a look at https://github.com/StefanScherer/consul-agent, an AppVeyor build badge is also there to see what happens in CI.

[StefanScherer]

In the meantime I pushed five versions and let AppVeyor build all the MSI's for the different Consul versions to show/test upgrades, blocking downgrades etc. Feedback is welcome ! ;-)

[carlpett]

Took some extra time due to work-related stuff, but here's our take: https://github.com/hashicorp/consul/pull/1640 There is no ui and such polish that @StefanScherer has in his version, though (we exclusively use this in automated scenarios, so it never came up). I'll try to do a more detailed comparison during the weekend to see if there are any other nice features that I can steal be inspired by.

[StefanScherer]

@carlpett Thanks for sharing! Looks pretty good.

Our setup with the package.json where we read many values from it comes from building MSI packages for Node.js microservices. We stripped that down to put Consul in a very similar MSI.

I also have removed the code signing steps we have to sign both the consul.exe as well as the final MSI file. This might be interesting for an official MSI build to suppress the "unknown publisher" Windows warnings.

A missing step are some tests with the MSI. I plan to add some at work and share them as well in the public repo soon.

[vassilvk]

@carlpett - this is a great setup! Thank you for sharing!

In my team we need a custom consul.json with a specific local bind_addr in place of the loopback advertise_addr. Do you think it is better to fork off your repo or to use it as a submodule in our repo?

Thanks, Vassil

[carlpett]

@vassilvk - Thanks! The service is installed with the -config-dir parameter pointing to the consul.d directory below the install directory. You should just be able to drop an additional configuration file there and have it working without modifying the installer.

[vassilvk]

Thanks, @carlpett. And @StefanScherer, thank you for open sourcing your WiX setup - this is really solid stuff!

[ricardclau]

Hi

We were having a similar conversation in one of the InfluxData Telegraf Github issues (https://github.com/influxdata/telegraf/issues/860) and @StianOvrevage mentioned https://godoc.org/bosun.org/cmd/scollector has a -winsvc flag which makes the binary register correctly as a Windows Service.

https://github.com/bosun-monitor/bosun/blob/master/cmd/scollector/service_windows.go seems to do the magic.

@slackpad @mitchellh Would you consider adding something similar to make running the Consul agent a bit easier in Windows machines? NSSM works great but it certainly seems a bit of a hack

[drusellers]

Any updates on this. Would love to have a known process for windows

[JonPaz]

Having recently come across this issue, we're using the Chocolatey supplied packge. Using Ansible to install the service and then set the appropriate config files (as a client). Works a treat!

https://chocolatey.org/packages/consul

[cdhunt]

It would be great if the golang.org/x/sys/windows/svc package was implemented to provide native Windows Service support w/o an additional binary dependency.

[pieterlouw]

By importing _ "github.com/kardianos/minwinsvc" into main.go will make the Windows binary installable as a Windows service using sc for example: sc create serviceName start= auto binPath= ""C:\path_to_app\app.exe" --commandLineArg arg1"

[StefanScherer]

Nice trick. Thanks @pieterlouw

[MagnumOpus21]

4244 Fixes this.