search

hashicorp / packer-plugin-proxmox

Packer plugin for Proxmox Builder
https://www.packer.io/docs/builders/proxmox
Mozilla Public License 2.0
163 stars 73 forks source link

storage format qcow2 unsupporter with packer 1.8.1-1 #92

Open lethargosapatheia opened 2 years ago

lethargosapatheia commented 2 years ago

Overview of the Issue

Hello,

It seems that packer errors out when trying to deploy a template and it complains about qcow2 being unsupported. So I thought I would start here rather than on the hashicorp forum, as I'm not really sure where the problem lies.

Reproduction Steps

Running packer build on a proxmox server generates this error. This isn't happening with packer 1.8.0.

Plugin and Packer version

v.18.1

Simplified Packer Buildfile

{
  "builders": [
    {
      "type": "proxmox",
      "proxmox_url": "https://{{ user `proxmox_host` }}:8006/api2/json",
      "username": "{{ user `proxmox_username` }}",
      "token": "{{ user `proxmox_token` }}",
      "node": "{{ user `proxmox_node_name` }}",
      "ssh_password": "{{ user `ssh_password` }}",
      "ssh_timeout": "30m",
      "ssh_username": "{{ user `ssh_username` }}",
      "template_name": "{{ user `template_name` }}",
      "template_description": "{{ user `template_description` }}",
      "sockets": 1,
      "cores": 4,
      "memory": 6144,
      "disks": [
        {
          "type": "scsi",
          "disk_size": "10G",
          "format": "qcow2",
          "storage_pool": "local-lvm",
          "storage_pool_type": "lvm-thin",
          "cache_mode": "none",
          "io_thread": "false"
        }
      ],
      "network_adapters": [
        {
          "bridge": "vmbr0",
          "model": "virtio"
        }
      ],
      "os": "l26",
      "unmount_iso": true,
      "cloud_init": true,
      "cloud_init_storage_pool": "cloudinit",
      "http_directory": "http",
      "http_interface": "ens160",
      "http_port_max": 8039,
      "http_port_min": 8039,
      "insecure_skip_tls_verify": false,
      "iso_checksum": "f8e3086f3cea0fb3fefb29937ab5ed9d19e767079633960ccb50e76153effc98",
      "iso_file": "nfs:iso/ubuntu-22.04-live-server-amd64.iso",
      "boot_wait": "4s",
      "boot_command": [
        "<e><bs><down><down><down>",
    "<right><right><right><right><right><right><right><right><right><right><right><right><right><right><right><right><right><right><right><right><right><right><right>",
        "<spacebar>",
        "ip={{ user `vm_ip` }}::{{ user `vm_gateway` }}:{{ user `vm_netmask` }}::::{{ user `vm_dns` }} ",
        "autoinstall 'ds=nocloud-net;s=http://{{ .HTTPIP }}:{{ .HTTPPort }}/' ",
        "<F10>"
      ]
    },
[...]
  "variables": {
    "vm_ip": "10.88.88.157",
    "vm_netmask": "255.255.255.0",
    "vm_gateway": "10.88.88.126",
    "vm_dns": "10.88.88.126",
    "search_domain:": "mydomain.com",
    "dns_server:": "10.88.88.126",
    "template_name": "packer-template",
    "template_description": "Ubuntu 22.04, generated by packer on {{ isotime }}",
    "playbook": "setup/work.yml",
    "ssh_password": "",
    "ssh_username": "",
    "vmware_build": "no",
    "vsphere_build_host": "",
    "vsphere_datastore": "",
    "vsphere_folder": "",
    "vsphere_cluster": "",
    "vsphere_network": "",
    "vsphere_password": "",
    "vsphere_username": ""
  }

Operating system and Environment details

Running packer on Ubuntu 20.04.4

Log Fragments and crash.log files

2022/06/05 16:16:55 [INFO] Packer version: 1.8.1 [go1.17.8 linux amd64]
2022/06/05 16:16:55 [TRACE] discovering plugins in /usr/bin
2022/06/05 16:16:55 [TRACE] discovering plugins in /root/.config/packer/plugins
2022/06/05 16:16:55 [TRACE] discovering plugins in .
2022/06/05 16:16:55 [INFO] PACKER_CONFIG env var not set; checking the default config file path
2022/06/05 16:16:55 [INFO] PACKER_CONFIG env var set; attempting to open config file: /root/.packerconfig
2022/06/05 16:16:55 [WARN] Config file doesn't exist: /root/.packerconfig
2022/06/05 16:16:55 [INFO] Setting cache directory: /root/.cache/packer
2022/06/05 16:16:55 [TRACE] Starting internal plugin packer-builder-proxmox
2022/06/05 16:16:55 Starting plugin: /usr/bin/packer []string{"/usr/bin/packer", "plugin", "packer-builder-proxmox"}
2022/06/05 16:16:55 Waiting for RPC address for: /usr/bin/packer
2022/06/05 16:16:55 packer-builder-proxmox plugin: [INFO] Packer version: 1.8.1 [go1.17.8 linux amd64]
2022/06/05 16:16:55 Received unix RPC address for /usr/bin/packer: addr is /tmp/packer-plugin2630202321
2022/06/05 16:16:55 packer-builder-proxmox plugin: [INFO] PACKER_CONFIG env var not set; checking the default config file path
2022/06/05 16:16:55 packer-builder-proxmox plugin: [INFO] PACKER_CONFIG env var set; attempting to open config file: /root/.packerconfig
2022/06/05 16:16:55 packer-builder-proxmox plugin: [WARN] Config file doesn't exist: /root/.packerconfig
2022/06/05 16:16:55 packer-builder-proxmox plugin: [INFO] Setting cache directory: /root/.cache/packer
2022/06/05 16:16:55 packer-builder-proxmox plugin: args: []string{"packer-builder-proxmox"}
2022/06/05 16:16:55 [TRACE] Starting internal plugin packer-provisioner-ansible
2022/06/05 16:16:55 Starting plugin: /usr/bin/packer []string{"/usr/bin/packer", "plugin", "packer-provisioner-ansible"}
2022/06/05 16:16:55 Waiting for RPC address for: /usr/bin/packer
2022/06/05 16:16:55 packer-builder-proxmox plugin: Plugin address: unix /tmp/packer-plugin2630202321
2022/06/05 16:16:55 packer-builder-proxmox plugin: Waiting for connection...
2022/06/05 16:16:55 packer-builder-proxmox plugin: Serving a plugin connection...
2022/06/05 16:16:55 packer-provisioner-ansible plugin: [INFO] Packer version: 1.8.1 [go1.17.8 linux amd64]
2022/06/05 16:16:55 packer-provisioner-ansible plugin: [INFO] PACKER_CONFIG env var not set; checking the default config file path
2022/06/05 16:16:55 packer-provisioner-ansible plugin: [INFO] PACKER_CONFIG env var set; attempting to open config file: /root/.packerconfig
2022/06/05 16:16:55 packer-provisioner-ansible plugin: [WARN] Config file doesn't exist: /root/.packerconfig
2022/06/05 16:16:55 packer-provisioner-ansible plugin: [INFO] Setting cache directory: /root/.cache/packer
2022/06/05 16:16:55 packer-provisioner-ansible plugin: args: []string{"packer-provisioner-ansible"}
2022/06/05 16:16:55 packer-provisioner-ansible plugin: Plugin address: unix /tmp/packer-plugin3530321557
2022/06/05 16:16:55 Received unix RPC address for /usr/bin/packer: addr is /tmp/packer-plugin3530321557
2022/06/05 16:16:55 packer-provisioner-ansible plugin: Waiting for connection...
2022/06/05 16:16:55 packer-provisioner-ansible plugin: Serving a plugin connection...
2022/06/05 16:16:55 Preparing build: proxmox
2022/06/05 16:16:55 packer-builder-proxmox plugin: CPU type not set, using default 'kvm64'
2022/06/05 16:16:55 packer-builder-proxmox plugin: SCSI controller not set, using default 'lsi'
proxmox: output will be in this color.
2022/06/05 16:16:56 Build debug mode: false
2022/06/05 16:16:56 Force build: false
2022/06/05 16:16:56 On error: abort
2022/06/05 16:16:56 Waiting on builds to complete...
2022/06/05 16:16:56 Starting build run: proxmox
2022/06/05 16:16:56 Running builder: proxmox

2022/06/05 16:16:56 [INFO] (telemetry) Starting builder proxmox
2022/06/05 16:16:56 packer-provisioner-ansible plugin: ansible-playbook version: 2.12.6
==> proxmox: Creating VM
2022/06/05 16:16:56 packer-builder-proxmox plugin: using token auth
2022/06/05 16:16:56 packer-builder-proxmox plugin: No URLs were provided to Step Download. Continuing...
==> proxmox: No VM ID given, getting next free from Proxmox
==> proxmox: Error creating VM: error creating VM: unable to create VM 112 - unsupported format 'qcow2' at /usr/share/perl5/PVE/Storage/LvmThinPlugin.pm line 87., error status:  (params: map[agent:1 args: boot: cores:4 cpu:kvm64 description:Packer ephemeral build VM hotplug: ide2:nfs:iso/ubuntu-22.04-live-server-amd64.iso,media=cdrom kvm:true machine: memory:6144 name:packer-629cd6f7-1a53-e158-3fa1-9631569995bb net0:virtio=FA:1A:ED:20:9B:76,bridge=vmbr0,firewall=false numa:false onboot:false ostype:l26 scsi0:local-lvm:10,format=qcow2,iothread=false scsihw:lsi sockets:1 tablet:false tags: vmid:112])
==> proxmox: Error creating VM: error creating VM: unable to create VM 112 - unsupported format 'qcow2' at /usr/share/perl5/PVE/Storage/LvmThinPlugin.pm line 87., error status:  (params: map[agent:1 args: boot: cores:4 cpu:kvm64 description:Packer ephemeral build VM hotplug: ide2:nfs:iso/ubuntu-22.04-live-server-amd64.iso,media=cdrom kvm:true machine: memory:6144 name:packer-629cd6f7-1a53-e158-3fa1-9631569995bb net0:virtio=FA:1A:ED:20:9B:76,bridge=vmbr0,firewall=false numa:false onboot:false ostype:l26 scsi0:local-lvm:10,format=qcow2,iothread=false scsihw:lsi sockets:1 tablet:false tags: vmid:112])
==> proxmox: Step "stepStartVM" failed, aborting...
2022/06/05 16:16:58 [INFO] (telemetry) ending proxmox
==> Wait completed after 2 seconds 111 milliseconds
==> proxmox: aborted: skipping cleanup of step "stepUploadISO"
==> proxmox: aborted: skipping cleanup of step "StepDownload"
2022/06/05 16:16:58 machine readable: error-count []string{"1"}
==> Some builds didn't complete successfully and had errors:
2022/06/05 16:16:58 machine readable: proxmox,error []string{"Error creating VM: error creating VM: unable to create VM 112 - unsupported format 'qcow2' at /usr/share/perl5/PVE/Storage/LvmThinPlugin.pm line 87., error status:  (params: map[agent:1 args: boot: cores:4 cpu:kvm64 description:Packer ephemeral build VM hotplug: ide2:nfs:iso/ubuntu-22.04-live-server-amd64.iso,media=cdrom kvm:true machine: memory:6144 name:packer-629cd6f7-1a53-e158-3fa1-9631569995bb net0:virtio=FA:1A:ED:20:9B:76,bridge=vmbr0,firewall=false numa:false onboot:false ostype:l26 scsi0:local-lvm:10,format=qcow2,iothread=false scsihw:lsi sockets:1 tablet:false tags: vmid:112])"}
==> Builds finished but no artifacts were created.
2022/06/05 16:16:58 [INFO] (telemetry) Finalizing.
Build 'proxmox' errored after 2 seconds 111 milliseconds: Error creating VM: error creating VM: unable to create VM 112 - unsupported format 'qcow2' at /usr/share/perl5/PVE/Storage/LvmThinPlugin.pm line 87., error status:  (params: map[agent:1 args: boot: cores:4 cpu:kvm64 description:Packer ephemeral build VM hotplug: ide2:nfs:iso/ubuntu-22.04-live-server-amd64.iso,media=cdrom kvm:true machine: memory:6144 name:packer-629cd6f7-1a53-e158-3fa1-9631569995bb net0:virtio=FA:1A:ED:20:9B:76,bridge=vmbr0,firewall=false numa:false onboot:false ostype:l26 scsi0:local-lvm:10,format=qcow2,iothread=false scsihw:lsi sockets:1 tablet:false tags: vmid:112])

==> Wait completed after 2 seconds 111 milliseconds

==> Some builds didn't complete successfully and had errors:
--> proxmox: Error creating VM: error creating VM: unable to create VM 112 - unsupported format 'qcow2' at /usr/share/perl5/PVE/Storage/LvmThinPlugin.pm line 87., error status:  (params: map[agent:1 args: boot: cores:4 cpu:kvm64 description:Packer ephemeral build VM hotplug: ide2:nfs:iso/ubuntu-22.04-live-server-amd64.iso,media=cdrom kvm:true machine: memory:6144 name:packer-629cd6f7-1a53-e158-3fa1-9631569995bb net0:virtio=FA:1A:ED:20:9B:76,bridge=vmbr0,firewall=false numa:false onboot:false ostype:l26 scsi0:local-lvm:10,format=qcow2,iothread=false scsihw:lsi sockets:1 tablet:false tags: vmid:112])

==> Builds finished but no artifacts were created.
2022/06/05 16:16:59 waiting for all plugin processes to complete...
2022/06/05 16:16:59 /usr/bin/packer: plugin process exited
2022/06/05 16:16:59 /usr/bin/packer: plugin process exited

I've tested this on both proxmox 6.4-14 and 7.2-4.

Reuenthal767 commented 2 years ago

On Archlinux I'm experiencing the same issue on packer-1.8.1-1-x86_64 and on packer 1.8.2-1-x86_64. The issue is not present with packer-1.8.0-2-x86_64.

Packer plugin does not seem to influence the triggering of the bug.

The culprit seems to be here: Error creating VM: error creating VM: unable to create VM 80000 - unsupported format 'qcow2' at /usr/share/perl5/PVE/Storage/LvmThinPlugin.pm line 87.

    84  sub alloc_image {
    85      my ($class, $storeid, $scfg, $vmid, $fmt, $name, $size) = @_;
    86
    87      die "unsupported format '$fmt'" if $fmt ne 'raw';
    88
    89      die "illegal name '$name' - should be 'vm-$vmid-*'\n"
    90          if  $name && $name !~ m/^vm-$vmid-/;
    91
    92      my $vgs = PVE::Storage::LVMPlugin::lvm_vgs();
    93
    94      my $vg = $scfg->{vgname};
    95
    96      die "no such volume group '$vg'\n" if !defined ($vgs->{$vg});
    97
    98      $name = $class->find_free_diskname($storeid, $scfg, $vmid)
    99          if !$name;
   100
   101      my $cmd = ['/sbin/lvcreate', '-aly', '-V', "${size}k", '--name', $name,
   102                 '--thinpool', "$vg/$scfg->{thinpool}" ];
   103
   104      run_command($cmd, errmsg => "lvcreate '$vg/$name' error");
   105
   106      return $name;
   107  }

As you can see from the path, this file lives on the Proxmox Server, so it didn't change between packer versions. Which means that for some reason the non-bugged packer versions were not calling that method.

nywilken commented 2 years ago

Thanks for reaching out. We will take a look at what might be happening here before the next release.

lethargosapatheia commented 2 years ago

Any updates on this issue? On version 1.8.3 I get the same error.

jerrac commented 2 years ago

Is qcow2 actually supposed to be supported?

I see https://github.com/hashicorp/packer-plugin-proxmox/issues/28 which mentions it is not supported, and https://github.com/hashicorp/packer-plugin-proxmox/issues/29 which is asking for support. Then there is https://github.com/hashicorp/packer-plugin-proxmox/issues/23 and this issue that make it sound like qcow2 support is expected. Plus, the docs list it as an option.

In any case, I need thin provisioning for my use case. And qcow2 is not working with Packer 1.8.3 and Proxmox Community 7.2-11. 1.0.6 or greater is the version of this plugin I'm on.

Anyone have any workarounds?

lethargosapatheia commented 2 years ago

Well, it's worked wthout any issues until this came up in this version, so in this sense it was definitely supported. And yes, given that it's an official option, you infer that it's also officially supported.

The only workaround for me was to keep pin the version to 1.8.0, which probably won't change anything as far as proxmox is concerned. I'm sure no one is hurrying to bring new features for it :) The focus is going to be the cloud, vsphere etc.

jerrac commented 2 years ago

Well I figured out my issue. I was not using a storage pool on Proxmox that supported vm images. Once I configured a "Directory" type, my builds started working. Both the old version of Packer, and the current 1.8.3 version.

53845714nF commented 1 year ago

You can also change the disk format to "raw". I don't know if this is a bad decision but it works.

lethargosapatheia commented 1 year ago

I have to say, the state of automatisation in proxmox and collaboration with the proxmox developers is deplorable. There's so much you just have to accept if you want to self-host things and use open source software. It is what it is.

ckdarby commented 1 year ago

I'm using Packer 1.8.5 same issue with qcow2. Switched to "raw" as mentioned by @53845714nF

Xp101T7 commented 1 year ago

I switch to raw and then I get

==> proxmox-iso.windows: Starting VM 2023/09/23 02:01:27 packer-plugin-proxmox_v1.1.3_x5.0_linux_amd64 plugin: 2023/09/23 02:01:27 No boot command given, skipping 2023/09/23 02:01:30 packer-plugin-proxmox_v1.1.3_x5.0_linux_amd64 plugin: 2023/09/23 02:01:30 [DEBUG] Unable to get address during connection step: 500 QEMU guest agent is not running 2023/09/23 02:01:30 packer-plugin-proxmox_v1.1.3_x5.0_linux_amd64 plugin: 2023/09/23 02:01:30 Waiting for WinRM, up to timeout: 30m0s ==> proxmox-iso.windows: Waiting for WinRM to become available... 2023/09/23 02:01:33 packer-plugin-proxmox_v1.1.3_x5.0_linux_amd64 plugin: 2023/09/23 02:01:33 [DEBUG] Error getting WinRM host: 500 QEMU guest agent is not running

Could this be related to QEMU raw settings not allowing QEMU guest agent to to run? I boot the host up manually after it creates the VM and then I get the callback error.

53845714nF commented 1 year ago

In my opinion, this has nothing to do with the file system format. See if you need to install qemu agent with packer or just turn it on. I have a qemu_agent = true in my setting. Maybe it has to be already installed in the ISO.

Here is my config as an example: 

source "proxmox-iso" "debian12" {
    # proxmox settings
    proxmox_url = "${var.proxmox_url}"
    node = "${var.node}"
    username = "${var.username}"
    token = "${var.token}" 
    insecure_skip_tls_verify = true

    # VM General Settings
    vm_id = "${var.vm_id}"
    vm_name = "debian12-template"
    template_description = "Debian 12 Server Image"

    iso_file = "${var.proxmox_iso_pool}/${var.debian_image}"
    unmount_iso = true

    qemu_agent = true
Xp101T7 commented 1 year ago

I figured it out the VM takes a long while to callback to the provisioner WinRM. When the WinRM provisioning host, dial tcp takes place it was not able to connect bc of the networking I had setup. The route to the host was not setup, so this was not an issue so far with changing format = "qcow2" to "raw". @53845714nF Thank you for helping me understand how the qemu agent is called in the script this made me go read through the config files more.

lethargosapatheia commented 1 year ago

Changing qemu_agent to true doesn't change anything in my case. This is saw disappointing, it's beyond words. They've actually updated packer and haven't checked if the disk format directive actually does what it's supposed to be doing.