AWS AMI - not syspreped

lcc2207 commented 9 years ago

I appears that the AMIs have not been syspreped, or am I missing how to sysprep them as part of the build?

dylanmei commented 9 years ago

There's an AWS example template in this repo: dylanmei/packer-windows-templates. EC2 User Data is used to run some basic setup on a new instance. Does that answer your question?

lcc2207 commented 9 years ago

Unless I am looking at the wrong sample, not really those setups are for setting up WinRM to get into the instance. What I am seeing is more there is no process to run ec2config or sysprep before making the instances an AMI. This results in all the Windows instances deployed from this AMI having the same SID etc.

timsutton commented 9 years ago

In the past I've tested running sysprep commands for doing vmware/vbox images - which works, but IIRC my issue was that the sysprep process shuts down the box, but the Packer builder is expecting that this is its responsibility, so it would hang there and ultimately time out or I'd abort it.

mefellows commented 9 years ago

In this case of Virtualbox/Parallels/VMWare you have the shutdown_command configuration item - I've used that in the past to execute sysprep with the shutdown option. This works nicely.

In the case of AWS this option is not available, however you can still control how the AMI export process behaves via the EC2Config service. It's fairly easy to do as it's just XML configuration, and it is in here that you can do this sort of thing.

Here is a Gist that:

Enables EC2 User Data to be executed on a boot of the AMI
Sets the computer name
Sets the password if a key is provided (for decryption using the CLI/Console and RDP)
Tells the EC2Config service to sysprep the machine when it is exported to an AMI

Ideally, you avoid sysprepping and domain-joining where possible - it's slow, and a bit messy with the whole cloud & auto-scaling thing, although I understand Amazon are making this easier. We have worked around this problem in other ways - IAM roles, security groups/Network ACLS, security tokens for API calls and so on. Hope this helps.

For now, I'm going to close this issue as I believe this documentation is sufficient to move forward.

timsutton commented 9 years ago

Thanks for the suggestion about using shutdown_command to do the sysprep - seems obvious in retrospect!

lcc2207 commented 9 years ago

Would it be possible to get the shutdown_command added to the AWS module?

mefellows commented 9 years ago

There is a PR that includes this feature. You could give that a shot whilst we work on integrating it (it's fairly big). So i suppose the answer is yes but if you're in a hurry I'd suggest running you're own script for now as per above. Hope that helps?

aalbertson commented 8 years ago

I would just caution having the "shutdown_command" making sure it can be overridden if you DON'T want it to sysprep. There are valid reasons in AWS where you might not, granted most people WILL want it, but it is faster (and less reboots) if you don't (and just use a statically created user account(s)).

ghost commented 8 years ago

@mefellows I followed your steps but the image is still not syspreped...

mefellows commented 8 years ago

Hi @rdmips, we're going to need more information if you would like some assistance.

Examples aside, please familiarise yourself with both http://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/UsingConfig_WinAMI.html#UsingConfigXML_WinAMI and http://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/ami-create-standard.html which explain the concepts and practices.

Packer (and this now mostly obsolete project) support this feature by providing a way to upload a file in tho correct location.

ghost commented 8 years ago

Sure.

Packer provisioner type : { "type" : "powershell", "elevated_user" : "{{user winrm_username }}", "elevated_password" : "{{user winrm_password }}", "scripts" : [ "./scripts/Ec2Config.ps1", "./scripts/BundleConfig.ps1"] }

BundleConfig.ps1 : $EC2SettingsFile="C:\Program Files\Amazon\Ec2ConfigService\Settings\BundleConfig.xml" $xml = [xml](get-content $EC2SettingsFile) $xmlElement = $xml.get_DocumentElement()

foreach ($element in $xmlElement.Property){ if ($element.Name -eq "AutoSysprep"){ $element.Value="Yes" } } $xml.Save($EC2SettingsFile)

EC2Config.ps1 $EC2SettingsFile="C:\Program Files\Amazon\Ec2ConfigService\Settings\Config.xml" $xml = [xml](get-content $EC2SettingsFile) $xmlElement = $xml.get_DocumentElement() $xmlElementToModify = $xmlElement.Plugins

foreach ($element in $xmlElementToModify.Plugin){ if ($element.name -eq "Ec2SetPassword"){ $element.State="Enabled" } elseif ($element.name -eq "Ec2SetComputerName"){ $element.State="Enabled" } elseif ($element.name -eq "Ec2HandleUserData"){ $element.State="Enabled" } } $xml.Save($EC2SettingsFile)

mefellows commented 8 years ago

Thanks @rdmips. First of all, how are you determining that the Sysprep has not been performed? (i.e. before after expectations).

WRT diagnosing, are you able to perform a Packer run with the -debug flag and take a snapshot of the modified BundleConfig.xml after the powershell provisioner has been run? Or even create another provisioner which cats it to stdout and therefore in the logs? It would be good to first see if it modifies the correct files for the EC2ConfigService.

ghost commented 8 years ago

@mefellows Yes I was able to run packer with debug and the BundleConfig.xml is being updated correctly. I also tried to build an image using the following git repository : https://gist.github.com/mefellows/a7a5ef98a3ec21a2c972

If you try to create a image from the git repository above and provision two or more servers, the Windows Machine SID will be the same. The command below provides the USER SIDs or https://technet.microsoft.com/en-us/sysinternals/bb897417.aspx PsGetSid MACHINE SID.

gwmi win32userprofile | select -unique @{name="Name";expression={$._server}},@{name="SID";expression={$.sid}},@{name="LastUseTime";expression={$.converttodatetime($.lastusetime)}},localpath | ft -auto

Packer Debug : ==> amazon-windows-ebs: Provisioning with shell script: ./scripts/Ec2Config.ps1 amazon-windows-ebs: ==> amazon-windows-ebs: Provisioning with shell script: ./scripts/BundleConfig.ps1 ==> amazon-windows-ebs: Pausing after run of step 'StepProvision'. Press enter to continue. ==> amazon-windows-ebs: Stopping the source instance... ==> amazon-windows-ebs: Waiting for the instance to stop...

Server provisioned using AMI built with Packer. AWS System Log : 2016/03/24 22:38:05Z: Windows sysprep configuration complete. 2016/03/24 22:38:06Z: Driver: AWS PV Storage Host Adapter v7.3.2.0 2016/03/24 22:38:06Z: Driver: Intel(R) 82599 Virtual Function v1.0.15.3 2016/03/24 22:38:06Z: AMI Origin Version: 2016.03.09 2016/03/24 22:38:06Z: AMI Origin Name: Windows_Server-2012-R2_RTM-English-64Bit-Base 2016/03/24 22:38:06Z: OS: Microsoft Windows NT 6.3.9600 2016/03/24 22:38:06Z: OsVersion: 6.3 2016/03/24 22:38:06Z: OsProductName: Windows Server 2012 R2 Standard 2016/03/24 22:38:06Z: OsBuildLabEx: 9600.18202.amd64fre.winblue_ltsb.160119-0600 2016/03/24 22:38:06Z: Language: en-US 2016/03/24 22:38:06Z: TimeZone: Coordinated Universal Time 2016/03/24 22:38:06Z: Offset: UTC 00:00:00 2016/03/24 22:38:06Z: EC2 Agent: Ec2Config service v3.14.786 2016/03/24 22:38:06Z: Message: Waiting for meta-data accessibility... 2016/03/24 22:38:06Z: Message: Meta-data is now available. 2016/03/24 22:38:07Z: AMI-ID: ami-13f4fb79 2016/03/24 22:38:07Z: Instance-ID: i-b9c64122 2016/03/24 22:38:07Z: Ec2SetPassword: Disabled 2016/03/24 22:38:07Z: RDPCERTIFICATE-SUBJECTNAME: IP-0AA8A20A 2016/03/24 22:38:07Z: RDPCERTIFICATE-THUMBPRINT: 6590461127F8580BAD50EFD42FE33B9E4635B21C 2016/03/24 22:38:14Z: Message: Windows is Ready to use

Thank you so much for your help.

ghost commented 8 years ago

@mefellows

==> amazon-windows-ebs: Provisioning with shell script: ./scripts/Ec2Config.ps1 ==> amazon-windows-ebs: Provisioning with shell script: ./scripts/BundleConfig.ps1

I had to logon to the instance and run EC2Config --sysprep before the step below : _==> amazon-windows-ebs: Pausing after run of step 'StepProvision'. Press enter to continue._ ==> amazon-windows-ebs: Stopping the source instance... ==> amazon-windows-ebs: Waiting for the instance to stop... ==> amazon-windows-ebs: Pausing after run of step 'stepStopInstance'. Press enter to continue. ==> amazon-windows-ebs: Pausing after run of step 'stepModifyInstance'. Press enter to continue.

mefellows commented 8 years ago

Interesting. So when you manually ran EC2Config --sysprep the SIDs were updated as expected? I suppose that wouldn't be difficult to automate, but my understanding was that wasn't required.

Upon review of it http://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/ami-create-standard.html#sysprep-using, it seems %ProgramFiles%\Amazon\Ec2ConfigService\ec2config.exe -sysprep might be required to run on shutdown. I'll likely be able to look into this properly early next week, but if you wanted to give that a try and report back that would be super!

ghost commented 8 years ago

Correct. For some reasons the AutoSysprep doesn't work.

On the bundleconfig.xml you change the switch /shutdown to /quit /oobe /quit /generalize so when sysprep runs it will not shutdown the server. Of course you run the EC2Config --sysprep the last of your commands because your RDP session will be terminated.

cd "C:\Program Files\Amazon\Ec2ConfigService\" .\Ec2Config.exe -sysprep

Packer will stop the instance and take an image.

fzk-rec commented 7 years ago

Thanks for the ideas/solutions @ghost. I had to solve the 'apply sysprep to an AMI through packer' problem today too.

I used @ghost's example code and have some additions:

in both your ps1 scripts, the syntax in line 2 is wrong, it should be $xml = [xml](get-content $EC2SettingsFile) . With your $xml = xml I receive exceptions from powershell
Here is a code example to exchange the default Sysprep switches from BundleConfig.xml with the ones that @ghost posted

On the bundleconfig.xml you change the switch /shutdown to /quit /oobe /quit /generalize

$EC2SettingsFile="C:\Program Files\Amazon\Ec2ConfigService\Settings\BundleConfig.xml" $xml = [xml](get-content $EC2SettingsFile)

$xml_switches = $xml.SelectSingleNode("//Switches[1]") foreach ($switch in $xml_switches) { $switch.'#text' = "/quit /oobe /generalize" } $xml.Save($EC2SettingsFile)

Extra info: The Ec2SetPassword from the config.xml does not generate a random password after sysprep (Instead it generates a random password on every bootup). The correct parameter for this is SetPasswordAfterSysprep in the BundleConfig.xml

So if you want to switch the sysprep parameters and want sysprep to generate a random password, you can use this: $EC2SettingsFile="C:\Program Files\Amazon\Ec2ConfigService\Settings\BundleConfig.xml" $xml = [xml](get-content $EC2SettingsFile)

$xml_switches = $xml.SelectSingleNode("//Switches[1]") foreach ($switch in $xml_switches) { $switch.'#text' = "/quit /oobe /generalize" } $xml.Save($EC2SettingsFile)

$xml = [xml](get-content $EC2SettingsFile) $xmlElement = $xml.get_DocumentElement() $xmlElementToModify = $xmlElement.Property foreach ($element in $xmlElementToModify.Property){ if ($element.Name -eq "SetPasswordAfterSysprep"){ $element.Value="Enabled" } }

Don't forget to execute the sysprep $p = Start-Process "C:\Program Files\Amazon\Ec2ConfigService\Ec2Config.exe" -ArgumentList "-sysprep" -Wait

dayglojesus commented 7 years ago

@fzk-rec Most concise set of instructions I have seen. Well done.

packer-community / packer-windows-plugins

AWS AMI - not syspreped #46