packer-community / packer-windows-plugins

A suite of Packer plugins for provisioning Windows machines
112 stars 20 forks source link

Long-running Windows2012 build applying updates times out when provisioning #44

Closed robcoward closed 9 years ago

robcoward commented 9 years ago

In trying to use the virtualbox-windows-iso builder in conjuction with the powershell provisioner, making use of the boxstarter powershell modules to work around the microsoft enforced restrictions on remotely installing windows update packages over winrm connections, I am hitting a 2 hour timeout error before the windows updates finishes applying and the whole build aborts.

I believe I have tracked the issue down to the following line of code: https://github.com/packer-community/packer-windows-plugins/blob/master/communicator/winrm/communicator.go#L57

Is there a reason for hard-coding a 2 hour timeout ? Could it be made a configurable option please ?

mefellows commented 9 years ago

Thanks @robcoward, there has been discussion on this topic so we sort of knew at some point we'd need to properly address this.

FYI another good place to apply updates is in the autounattend scripts (see Joe Fitzgeralds packer templates) which doesn't suffer from this problem.

Either way we'll sort out the timeout issue, probably by making it configurable.

robcoward commented 9 years ago

@mefellows making it configurable would suit me. I'm currently working around the problem by using the virtualbox-iso builder to churn out an OVA file with most updates applied since the ssh connection doesnt have the same timeout limit, and then your virtualbox-windows-ovf build to finish the job, completing remaining updates and using the reboot-provisioner where needed.

I know it's still only release candidate, but loving the work you guys have done to allow us to use packer with native winrm instead of having to install openssh on a windows server.

mefellows commented 9 years ago

For now, pending discussion over at #26, I will set the timeout to a sufficiently stupendous and only slightly less arbitrary 24h which should cover most scenarios, one would have thought. This will get @robcoward out of a bind and buy us some time to think on a proper solution.

Also, thanks for the feedback Rob - it is much appreciated!

mefellows commented 9 years ago

To keep in alignment with the SSH communicator (from upstream Packer), a 24 hour timeout is our response to keeping the operation alive 'forever' and avoiding the need for another configuration item. For now, closing this one off.

robcoward commented 9 years ago

Hi @mefellows - thanks for increasing the timeout. I've downloaded the v1.0.0 linux_amd64.zip release in order to retry the long running windows update. The extracted files are timestamped Apr 28, so definitely after commit packer-community/packer-windows-plugins@dd01a98a03273ab474957ebfe1b299814e8ca8af and yet I'm still getting an error exactly 2 hours after the start of a powershell provisioner running windows update. This is the provisioner config:

        {           
            "type": "powershell",            
            "elevated_user": "{{user `winrm_userid`}}",
            "elevated_password": "{{user `winrm_password`}}",  
            "inline": [
                "Import-Module Boxstarter.WinConfig",
                "Install-WindowsUpdate -acceptEula -suppressReboots"
            ]
        },  

and the last few lines of packer output (piped through awk to prefix output lines with a timestamp):

2015-05-08 12:58:07 ^[[0;32m    VBoxWin2012: + Boxstarter finished Install 77 of 127 updates: Security Update for Microsoft .NET Framework 4.5, 4.5.1 and 4.5.2 on Windows 8 and Windows Server 2012 x64 (KB2979577) 00:00:41.0934128^[[0m
2015-05-08 12:58:07 ^[[0;32m    VBoxWin2012: + Boxstarter starting Download of Update for Windows Server 2012 (KB2995387)^[[0m
2015-05-08 12:59:20 ^[[0;32m    VBoxWin2012: + Boxstarter finished Download of Update for Windows Server 2012 (KB2995387) 00:01:12.5209520^[[0m
2015-05-08 12:59:20 ^[[0;32m    VBoxWin2012: + Boxstarter starting Install 78 of 127 updates: Update for Windows Server 2012 (KB2995387)^[[0m
2015-05-08 13:00:20 ^[[1;32m==> VBoxWin2012: Unregistering and deleting virtual machine...^[[0m
2015-05-08 13:00:23 ^[[1;32m==> VBoxWin2012: Deleting output directory...^[[0m
2015-05-08 13:00:23 ^[[1;31mBuild 'VBoxWin2012' errored: Script exited with non-zero exit status: 2147942659. Allowed exit codes are: [%!s(int=0)]^[[0m
2015-05-08 13:00:23
2015-05-08 13:00:23 ==> Some builds didn't complete successfully and had errors:
2015-05-08 13:00:23 --> VBoxWin2012: Script exited with non-zero exit status: 2147942659. Allowed exit codes are: [%!s(int=0)]
2015-05-08 13:00:23
2015-05-08 13:00:23 ==> Builds finished but no artifacts were created.

Any thoughts on what else might be causing this error after 2 hours of execution ? Is there anything else I can try ? Thanks Rob

mefellows commented 9 years ago

I think I've spotted the real culprit - https://github.com/packer-community/packer-windows-plugins/blob/master/provisioner/powershell/elevated.go (look for 2H). I'm not in front of a computer at the moment but will update some time over this weekend to 24h as per the other commit. If you're comfortable feel free to patch and submit a PR. I reckon that'll be it though. Sorry i missed it the first time.

robcoward commented 9 years ago

I'm happy to make the change and create a PR, however I don't currently have a working environment to compile go and create a replacement packer-provisioner-powershell to test.

mefellows commented 9 years ago

There you go: https://github.com/packer-community/packer-windows-plugins/releases/tag/v1.0.1-latest