Winrm connection not getting established while ami build

[tejasp04]

Issue : We are getting below error when we try to build AMI using packer .

Error logs: 2021/03/05 12:51:20 packer-builder-amazon-ebs plugin: [DEBUG] connecting to remote shell using WinRM 2021/03/05 12:51:50 packer-builder-amazon-ebs plugin: [ERROR] connection error: unknown error Post "https://13.211.157.185:5986/wsman": dial tcp 13.211.157.185:5986: i/o timeout 2021/03/05 12:51:50 packer-builder-amazon-ebs plugin: [ERROR] WinRM connection err: unknown error Post "https://13.211.157.185:5986/wsman": dial tcp 13.211.157.185:5986: i/o timeout 2021/03/05 12:51:55 packer-builder-amazon-ebs plugin: Using host value: 13.211.157.185 2021/03/05 12:51:55 packer-builder-amazon-ebs plugin: [INFO] Attempting WinRM connection...

**Packer Version** :  1.6.4

**Packer build file** : 

{
    "_intro": "This is a Packer (https://packer.io) template",
    "variables": {
      "git_sha": "{{env `GIT_SHA`}}",
      "ansible_skip_tags": "non_mutable",
      "ansible_verbose": "-vvv",
      "config_home": "{{env `APP_CONFIG_HOME`}}",
      "ansible_home": "{{env `ANSIBLE_HDIR`}}",
      "winrm_port": "{{env `PACKER_LISTN_PORT`}}",
      "winrm_user": "{{env `PACKER_USER`}}",
      "aws_region": "{{env `AWS_REGION`}}",
      "aws_account_type": "{{env `AWS_ACCOUNT_TYPE`}}",
      "app_parent": "{{env `APP_PARENT`}}",
      "app_name": "{{env `APP_NAME`}}",
      "app_env": "{{env `ENVIRONMENT`}}",
      "ami_prefix": "{{env `AMI_PREFIX`}}",
      "build_ip": "{{env `BUILD_IP`}}",
      "iam_profile": "{{env `IAM_PROFILE`}}",
      "os": "{{env `OS`}}"
    },
    "builders": [{
      "type": "amazon-ebs",
      "ami_name": "{{user `app_env`}}-{{user `app_parent`}}-{{user `app_name`}}-{{user `os`}}-CISBenchmark-{{user `git_sha`}}",
      "instance_type": "t2.large",
      "spot_price": "auto",
      "spot_price_auto_product": "Windows",
      "region": "{{user `aws_region`}}",
      "ami_description": "Using Base AMI {{ .SourceAMIName }}",
      "force_deregister": true,
      "force_delete_snapshot": true,
      "communicator": "winrm",
      "winrm_timeout": "12h",
      "winrm_username": "{{user `winrm_user`}}",
      "winrm_port": "{{user `winrm_port`}}",
      "winrm_use_ssl": true,
      "winrm_insecure": true,
      "winrm_use_ntlm": true,
      "user_data_file": "{{ template_dir  }}/win-packer-CIS-user-data.ps1",
      "subnet_filter": {
        "filters": {
          "tag:Type": "public"
        },
        "most_free": true,
        "random": true
      },
      "temporary_security_group_source_cidrs": "{{user `build_ip`}}",
      "associate_public_ip_address": true,
      "iam_instance_profile": "{{user `iam_profile`}}",
      "source_ami_filter": {
        "filters": {
            "name": "{{user `ami_prefix`}}*"
          },
          "owners": ["679593333241"],
          "most_recent": true
      },
      "run_tags": {
        "CommitID": "{{user `git_sha`}}",
        "Name": "packer-{{user `app_name`}}",
        "Family": "{{user `app_parent`}}",
        "Environment": "{{user `app_env`}}"      
      },
      "tags": {
        "Name": "{{user `app_env`}}-{{user `app_parent`}}-{{user `app_name`}}-{{user `os`}}-CISBenchmark-{{user `git_sha`}}",
        "OS_Version": "Windows-Server-2016",
        "Base_AMI_Name": "{{ .SourceAMIName }}",
        "Application": "{{user `app_name`}}",
        "Family": "{{user `app_parent`}}",
        "Environment": "{{user `app_env`}}",
        "ECS": "False",
        "Version": "latest"
      }
    }],
    "provisioners": [
      {
      "type": "shell-local",
      "command": "set -x; echo '[{{user `app_env`}}]' > {{user `config_home`}}/inv/packer.tmp && sed \"1s/\\$/ ansible_host=$(aws ec2 describe-instances --filter Name=tag:CommitID,Values=\"{{user `git_sha`}}\" Name=instance-state-name,Values=running | jq -r '.Reservations[].Instances[].PublicIpAddress')/\" < {{user `config_home`}}/inv/packer >> {{user `config_home`}}/inv/packer.tmp; cat {{user `config_home`}}/inv/packer.tmp ; cat {{user `config_home`}}/inv/packer"
      },
      {
        "type": "shell-local",
        "command": "ansible-playbook {{user `ansible_verbose`}} -i {{user `config_home`}}/inv/packer.tmp --connection=$ANSIBLE_CONNECTION_TYPE --extra-vars='targethosts={{user `app_env`}} ansible_user={{ user `winrm_user` }} ansible_password={{ .WinRMPassword }} ansible_winrm_transport=ntlm ansible_port={{ user `winrm_port` }} ansible_winrm_server_cert_validation=ignore ansible_winrm_operation_timeout_sec=240 ansible_winrm_read_timeout_sec=280' {{user `ansible_home`}}/shared-service-bake-ami.yml"
      },
      {
        "type": "powershell",
        "inline": [
          "C:\\ProgramData\\Amazon\\EC2-Windows\\Launch\\Scripts\\SendWindowsIsReady.ps1 -Schedule",
          "C:\\ProgramData\\Amazon\\EC2-Windows\\Launch\\Scripts\\InitializeInstance.ps1 -Schedule",
          "C:\\ProgramData\\Amazon\\EC2-Windows\\Launch\\Scripts\\InitializeDisks.ps1 -Schedule",
          "C:\\ProgramData\\Amazon\\EC2-Windows\\Launch\\Scripts\\SysprepInstance.ps1 -NoShutdown"
        ]
      }
    ]
  }

**User data ps1 file is as below** :

<powershell>

# MAKE SURE IN YOUR PACKER CONFIG TO SET:
#
#
#    "winrm_username": "Administrator",
#    "winrm_insecure": true,
#    "winrm_use_ssl": true,
#
#

write-output "Running User Data Script"
write-host "(host) Running User Data Script"

Set-ExecutionPolicy Unrestricted -Scope LocalMachine -Force -ErrorAction Ignore

# Don't set this before Set-ExecutionPolicy as it throws an error
$ErrorActionPreference = "stop"

# Remove HTTP listener
Remove-Item -Path WSMan:\Localhost\listener\listener* -Recurse

# Create a self-signed certificate to let ssl work
$Cert = New-SelfSignedCertificate -CertstoreLocation Cert:\LocalMachine\My -DnsName "packer"
New-Item -Path WSMan:\LocalHost\Listener -Transport HTTPS -Address * -CertificateThumbPrint $Cert.Thumbprint -Force

# WinRM
write-output "Setting up WinRM"
write-host "(host) setting up WinRM"

winrm quickconfig -q
winrm set "winrm/config" '@{MaxTimeoutms="1800000"}'
winrm set "winrm/config/winrs" '@{MaxMemoryPerShellMB="1024"}'
winrm set "winrm/config/service" '@{AllowUnencrypted="true"}'
winrm set "winrm/config/client" '@{AllowUnencrypted="true"}'
winrm set "winrm/config/service/auth" '@{Basic="true"}'
winrm set "winrm/config/client/auth" '@{Basic="true"}'
winrm set "winrm/config/service/auth" '@{CredSSP="true"}'
winrm set "winrm/config/listener?Address=*+Transport=HTTPS" "@{Port=`"5986`";Hostname=`"packer`";CertificateThumbprint=`"$($Cert.Thumbprint)`"}"
netsh advfirewall firewall set rule group="remote administration" new enable=yes
netsh firewall add portopening TCP 5986 "Port 5986"
stop-service winrm
set-service -name winrm -startuptype Automatic
start-service winrm

</powershell>

**OS version :** 

CIS Microsoft Windows Server 2016 Benchmark v1.2.* - Level 2"

Would you need anymore details ? 

[tejasp04]

Hi , can some one pls assist on it

[SwampDragons]

I'm not sure I've ever gotten a winrm connection to a CIS benchmark 2 server. That said, the issue is that a hardened image is explicitly meant to prevent unauthorized connections. Here's some context for you that may be helpful.

https://github.com/hashicorp/packer/issues/8093#issuecomment-530079184 https://github.com/hashicorp/packer/issues/9110 https://github.com/test-kitchen/kitchen-ec2/issues/408

I'd recommend downloading the actual checklist of security changes made to windows servers, and look over the ones that have to do with remote connections.

If I had to guess, I'd suspect that a level 2 benchmark may not like self-signed certificates.

[d2k-klin]

we are experiencing the same issue even with the CIS level 1: CIS Microsoft Windows Server 2016 Benchmark v1.2.0.7 - Level 1-* , everything worked fine until version v1.2.0.6, but after just changing the version to v1.2.0.7, started getting the error about WinRM timeout `2021/04/29 11:37:26 packer-builder-amazon-ebs plugin: Using host value: 18.184.205.217

2021/04/29 11:37:26 packer-builder-amazon-ebs plugin: [INFO] Attempting WinRM connection...

2021/04/29 11:37:26 packer-builder-amazon-ebs plugin: [DEBUG] connecting to remote shell using WinRM

2021/04/29 11:37:56 packer-builder-amazon-ebs plugin: [ERROR] connection error: unknown error Post "https://18.184.205.217:5986/wsman": dial tcp 18.184.205.217:5986: i/o timeout

2021/04/29 11:37:56 packer-builder-amazon-ebs plugin: [ERROR] WinRM connection err: unknown error Post "https://18.184.205.217:5986/wsman": dial tcp 18.184.205.217:5986: i/o timeout ` packer version: v1.7.0 ( also tried with v1.7.2(latest) -> same behaviour ) instance type : t2.xlarge configuration is almost the same as mentioned above by @tejasp04

We have also approached to CIS support about the changes in the latest version (v1.2.0.7), and here is the official answer from their side : "After some more testing on our end with the Windows Server 2016 Level 1 (v1.2.0.7) image, everything seems to work as expected for us. We are able to WinRM the same as before successfully."

PS: with simple revert to previous version v1.2.0.6 everything works fine, but this is not an option for us because we need the latest patches...

[ozbillwang]

any updates for this issue?

I hit this issue as well now.

[maffydavison]

II can start this again, since we have winrm problems with our builder -- just hanging. Our AMI is for windows 2019

[meyuviofficial]

I'm also running into a similar trouble. I'm trying to create an image out of the CIS Hardened Windows Server 2022. However, I am able to establish a connection with my userdata bootstrap script. But, the caveat is that I'm losing the connect after a reboot.