search

hashicorp / packer

Packer is a tool for creating identical machine images for multiple platforms from a single source configuration.
http://www.packer.io
Other
15.1k stars 3.33k forks source link

Build 'vmware-iso' errored: Error starting VM: VMware error: Error: The operation was canceled #5754

Closed basictheprogram closed 6 years ago

basictheprogram commented 6 years ago

$ uname -a Linux uart 4.4.0-103-generic #126-Ubuntu SMP Mon Dec 4 16:23:28 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux


- Debug log output from `PACKER_LOG=1 packer build template.json`.
  Please paste this in a gist https://gist.github.com

https://gist.github.com/basictheprogram/a2f157ad7fcedfe768f8a374dc614e23
SwampDragons commented 6 years ago

Thanks for reporting an issue; do you have simple steps to reproduce? Does this happen every time or only occasionally?

basictheprogram commented 6 years ago 
{
  "builders": [
    {
      "boot_wait": "0s",
      "communicator": "winrm",
      "cpu": 2,
      "disk_size": "{{user `disk_size`}}",
      "enable_secure_boot": true,
      "floppy_files": [
        "{{user `autounattend`}}",
        "./scripts/disable-winrm.ps1",
        "./scripts/enable-winrm.ps1",
        "./scripts/microsoft-updates.bat",
        "./scripts/win-updates.ps1"
      ],
      "guest_additions_mode": "disable",
      "iso_checksum": "{{user `iso_checksum`}}",
      "iso_checksum_type": "{{user `iso_checksum_type`}}",
      "iso_url": "{{user `iso_url`}}",
      "ram_size": 2048,
      "shutdown_command": "shutdown /s /t 10 /f /d p:4:1 /c \"Packer Shutdown\"",
      "switch_name": "{{user `hyperv_switchname`}}",
      "type": "hyperv-iso",
      "vm_name": "HyperVServer2016",
      "winrm_password": "vagrant",
      "winrm_timeout": "{{user `winrm_timeout`}}",
      "winrm_username": "vagrant"
    },
    {
      "boot_wait": "2m",
      "communicator": "winrm",
      "disk_size": "{{user `disk_size`}}",
      "disk_type_id": "{{user `disk_type_id`}}",
      "floppy_files": [
        "{{user `autounattend`}}",
        "./scripts/disable-winrm.ps1",
        "./scripts/enable-winrm.ps1",
        "./scripts/microsoft-updates.bat",
        "./scripts/win-updates.ps1"
      ],
      "guest_os_type": "windows9srv-64",
      "headless": "{{user `headless`}}",
      "iso_checksum": "{{user `iso_checksum`}}",
      "iso_checksum_type": "{{user `iso_checksum_type`}}",
      "iso_url": "{{user `iso_url`}}",
      "shutdown_command": "shutdown /s /t 10 /f /d p:4:1 /c \"Packer Shutdown\"",
      "type": "vmware-iso",
      "vmx_data": {
        "RemoteDisplay.vnc.enabled": "false",
        "RemoteDisplay.vnc.port": "5900",
        "memsize": "2048",
        "numvcpus": "2",
        "scsi0.virtualDev": "lsisas1068"
      },
      "vmx_remove_ethernet_interfaces": true,
      "vnc_port_max": 5980,
      "vnc_port_min": 5900,
      "winrm_password": "vagrant",
      "winrm_timeout": "{{user `winrm_timeout`}}",
      "winrm_username": "vagrant"
    },
    {
      "boot_wait": "2m",
      "communicator": "winrm",
      "disk_size": "{{user `disk_size`}}",
      "floppy_files": [
        "{{user `autounattend`}}",
        "./scripts/disable-winrm.ps1",
        "./scripts/enable-winrm.ps1",
        "./scripts/microsoft-updates.bat",
        "./scripts/win-updates.ps1"
      ],
      "guest_additions_mode": "disable",
      "guest_os_type": "Windows2012_64",
      "headless": "{{user `headless`}}",
      "iso_checksum": "{{user `iso_checksum`}}",
      "iso_checksum_type": "{{user `iso_checksum_type`}}",
      "iso_url": "{{user `iso_url`}}",
      "shutdown_command": "shutdown /s /t 10 /f /d p:4:1 /c \"Packer Shutdown\"",
      "type": "virtualbox-iso",
      "vboxmanage": [
        [
          "modifyvm",
          "{{.Name}}",
          "--memory",
          "2048"
        ],
        [
          "modifyvm",
          "{{.Name}}",
          "--cpus",
          "2"
        ]
      ],
      "winrm_password": "vagrant",
      "winrm_timeout": "{{user `winrm_timeout`}}",
      "winrm_username": "vagrant"
    }
  ],
  "post-processors": [
    {
      "keep_input_artifact": false,
      "output": "windows_2016_hyperv_{{.Provider}}.box",
      "type": "vagrant",
      "vagrantfile_template": "vagrantfile-windows_2016.template"
    }
  ],
  "provisioners": [
    {
      "scripts": [
        "./scripts/vm-guest-tools.bat",
        "./scripts/enable-rdp.bat"
      ],
      "type": "windows-shell"
    },
    {
      "scripts": [
        "./scripts/debloat-windows.ps1"
      ],
      "type": "powershell"
    },
    {
      "scripts": [
        "./scripts/set-winrm-automatic.bat",
        "./scripts/uac-enable.bat",
        "./scripts/compile-dotnet-assemblies.bat",
        "./scripts/dis-updates.bat",
        "./scripts/compact.bat"
      ],
      "type": "windows-shell"
    }
  ],
  "variables": {
    "autounattend": "./answer_files/2016_hyperv/Autounattend.xml",
    "disk_size": "61440",
    "disk_type_id": "1",
    "headless": "false",
    "iso_checksum": "53e2f01dc4077192a85f60f8d2ffb02189074e19b25f990cbe9eb767328d3fb6",
    "iso_checksum_type": "sha256",
    "iso_url": "http://care.dlservice.microsoft.com/dl/download/8/8/6/886DAAEF-81A7-4418-82D5-07D33B816962/14393.0.161119-1705.RS1_REFRESH_SERVERHYPERCORE_OEM_X64FRE_EN-US.ISO",
    "winrm_timeout": "6h"
  }
}
arizvisa commented 6 years ago

@basictheprogram, I'm doing a little bit of work on the vmware-iso builder. Do you have the .vmx for this build available, and what version of VMware Player are you using?

This'd probably be in output-vmware-iso/WindowsServer2016.vmx according to your log. If it gets deleted (which'd be surprising because packer has issues w/ cleanup during errors in VMware as a result of a handle leak), you can run packer with the -on-error=abort parameter and I think that should leave the file so you can snag it.

That .vmx will be key in reproducing the VMware issue that you're having.

basictheprogram commented 6 years ago

VMware Workstation 14.1.1 Player for Linux

$ ls -ld output ls: cannot access 'output': No such file or directory

Attempting a build with the -on-error=abort

Updated the gist at https://gist.github.com/basictheprogram/a2f157ad7fcedfe768f8a374dc614e23

The .vmx is below

#!/usr/bin/vmware
.encoding = "UTF-8"
bios.bootorder = "hdd,CDROM"
checkpoint.vmstate = ""
cleanshutdown = "TRUE"
config.version = "8"
displayname = "WindowsServer2016"
ehci.pcislotnumber = "34"
ehci.present = "FALSE"
ethernet0.addresstype = "generated"
ethernet0.bsdname = "en0"
ethernet0.connectiontype = "nat"
ethernet0.displayname = "Ethernet"
ethernet0.linkstatepropagation.enable = "FALSE"
ethernet0.pcislotnumber = "33"
ethernet0.present = "TRUE"
ethernet0.virtualdev = "e1000"
ethernet0.wakeonpcktrcv = "FALSE"
extendedconfigfile = "WindowsServer2016.vmxf"
floppy0.filename = "/tmp/packer809263452"
floppy0.filetype = "file"
floppy0.present = "TRUE"
guestos = "windows9srv-64"
gui.fullscreenatpoweron = "FALSE"
gui.viewmodeatpoweron = "windowed"
hgfs.linkrootshare = "TRUE"
hgfs.maprootshare = "TRUE"
ide1:0.devicetype = "cdrom-image"
ide1:0.filename = "/home/tanner/projects/packer/Stefan-Schere-packer-windows/packer_cache/49f719e23c56a779a991c4b4ad1680b8363918cd0bfd9ac6b52697d78a309855.iso"
ide1:0.present = "TRUE"
isolation.tools.hgfs.disable = "FALSE"
memsize = "2048"
msg.autoanswer = "true"
numvcpus = "2"
nvram = "WindowsServer2016.nvram"
pcibridge0.pcislotnumber = "17"
pcibridge0.present = "TRUE"
pcibridge4.functions = "8"
pcibridge4.pcislotnumber = "21"
pcibridge4.present = "TRUE"
pcibridge4.virtualdev = "pcieRootPort"
pcibridge5.functions = "8"
pcibridge5.pcislotnumber = "22"
pcibridge5.present = "TRUE"
pcibridge5.virtualdev = "pcieRootPort"
pcibridge6.functions = "8"
pcibridge6.pcislotnumber = "23"
pcibridge6.present = "TRUE"
pcibridge6.virtualdev = "pcieRootPort"
pcibridge7.functions = "8"
pcibridge7.pcislotnumber = "24"
pcibridge7.present = "TRUE"
pcibridge7.virtualdev = "pcieRootPort"
powertype.poweroff = "soft"
powertype.poweron = "soft"
powertype.reset = "soft"
powertype.suspend = "soft"
proxyapps.publishtohost = "FALSE"
remotedisplay.vnc.enabled = "TRUE"
remotedisplay.vnc.ip = "127.0.0.1"
remotedisplay.vnc.password = "eEVZcL0M"
remotedisplay.vnc.port = "5921"
replay.filename = ""
replay.supported = "FALSE"
scsi0.pcislotnumber = "16"
scsi0.present = "TRUE"
scsi0.virtualdev = "lsisas1068"
scsi0:0.filename = "disk.vmdk"
scsi0:0.present = "TRUE"
scsi0:0.redo = ""
sound.startconnected = "FALSE"
tools.synctime = "TRUE"
tools.upgrade.policy = "upgradeAtPowerCycle"
usb.pcislotnumber = "32"
usb.present = "FALSE"
uuid.action = "create"
virtualhw.productcompatibility = "hosted"
virtualhw.version = "12"
vmci0.id = "1861462627"
vmci0.pcislotnumber = "35"
vmci0.present = "TRUE"
vmotion.checkpointfbsize = "65536000"
RemoteDisplay.vnc.key = "HhwqDAEnMjYiCygZFysNDxcJNQA3NAY7JwMzGgM4MwYVMRQ9FSwoHh8SHjAMOBMoFjYoHwkNFxkYEgMJLSUKOzMKDzAyDR0DMAM5BTceER0zKycREDQiNw0iFy4XHAUuGiUcFgQ8CxQfNgYYBQA9GQomIyUNLgcYMwg9Ni8SGgA="
floppy0.clientDevice = "FALSE"
arizvisa commented 6 years ago

Hmm...you're using VMware workstation?

So, the issue is that packer's VMware driver is detecting your VMware instance as VMware player which means you probably have both installed. Is that the case? If you want to avoid rebuilding packer, you can uninstall vmware player and then things should work out.

2018/01/01 18:33:23 packer: 2018/01/01 18:33:23 Executing: /usr/bin/vmrun -T player start output-vmware-iso/WindowsServer2016.vmx nogui
...
2018/01/01 18:33:28 packer: 2018/01/01 18:33:28 Executing: /usr/bin/vmrun -T player list

I'm pretty certain that's what's causing the issue, but for the sake of completion, can you run /usr/bin/vmrun -T ws start output-vmware-iso/WindowsServer2016.vmx nogui instead of the one with -T player? If that works then the driver detection on linux can probably be tightened a bit..

arizvisa commented 6 years ago

Oh duh. Sorry, I failed at reading. I read "VMware Workstation" and read that as you were using VMware Workstation and not "VMware Workstation Player"..

basictheprogram commented 6 years ago

So packer and "Player" are not a supported configuration?

$ /usr/bin/vmrun -T player start output-vmware-iso/WindowsServer2016.vmx nogui
Error: The operation was canceled
$ /usr/bin/vmrun -T ws start output-vmware-iso/WindowsServer2016.vmx nogui
Error: The operation was canceled
arizvisa commented 6 years ago

Ok. So packer is pretty much doing everything as intended, and so...assuming you're on a box that supports more than 2 cpus, the issue is likely within VMware. Inside the output-vmware-iso/ directory should be a vmware.log file that gives more info.

arizvisa commented 6 years ago

In response to your comment, nah. They're supported. I just misread your prior comment where you mentioned "VMware Workstation Player" and interpreted that was "VMware Workstation" which is where my statement about that the difference between player and workstation came from.

I'm just never heard anybody refer to VMware player as "VMware Workstation Player". But it's all good..

You can post or review that .log file if you want me to still help tho. That should give insight into why vmrun is throwing that error.

basictheprogram commented 6 years ago

In response to your comment, nah. They're supported. I just misread your prior comment where you mentioned "VMware Workstation Player" and interpreted that was "VMware Workstation" which is where my statement about that the difference between player and workstation came from.

I'm just never heard anybody refer to VMware player as "VMware Workstation Player". But it's all good..

Cut-n-paste from my.vmware.com http://my.vmware.com/ download area.

You can post or review that .log file if you want me to still help tho. That should give insight into why vmrun is throwing that error.

2018-01-24T20:55:40.222-06:00| vmx| I125: Msg_Post: Error 2018-01-24T20:55:40.222-06:00| vmx| I125: [msg.cpuid.legacyCPU.nohwmmu.intel] This host does not support "Intel EPT" hardware assisted MMU virtualization. 2018-01-24T20:55:40.222-06:00| vmx| I125: [msg.cpuid.legacyCPU.noRealModeHV] This host does not support virtualizing real mode. The Intel "VMX Unrestricted Guest" feature is necessary to run this virtual machine on an Intel processor. 2018-01-24T20:55:40.222-06:00| vmx| I125: [msg.moduletable.powerOnFailed] Module 'CPUIDEarly' power on failed. 2018-01-24T20:55:40.222-06:00| vmx| I125: [msg.vmx.poweron.failed] Failed to start the virtual machine

Interesting! Virtualbox on the same computer works as expected.

vmware.log

arizvisa commented 6 years ago

So, docs and things suggest that you can specify monitor.allowLegacyCPU = "true" in the .vmx (I think there's some packer option that lets you manually add things to your .vmx).

This will drastically hit you performance-wise, as VMware will emulate all privileged instructions that get executed and be forced to use software for doing address translations (MMU), but if you're just building templates with packer you're probably just going to set-it-and-forget it so time shouldn't be too important..

Just make sure that you remove that option before you deploy them so none of your users have to suffer the perf hit. ;)

basictheprogram commented 6 years ago

Computer is remote, I'll check the BIOS tomorrow to make sure this isn't something I can turn on. If not, look for an appropriate computer. :-)

Thanks for the help. Learn a lot today. Appreciate it.

arizvisa commented 6 years ago

Apparently you can also downgrade VMware back to 12 as well according to this awesome guy, continuum: https://communities.vmware.com/thread/572961

np. gl.

basictheprogram commented 6 years ago

Link to download VMware Workstation 12.5.9 Player for Linux 64-bit' Link to download VMware VIX 1.15.8 Link to download vmware-vdiskmanager(doesn't work for me on Ubuntu 16.04) 

$ sudo apt-get install qemu-utils

Works for me.

arizvisa commented 6 years ago

@swampdragons, this issue looks like it can be closed as it was resolved to being a VMware issue with virtualization on the reporter's system.

ghost commented 4 years ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.