cloudinit: IDE controllers are unsupported for this QEMU binary or machine type

jouve commented 4 years ago

System Information

Linux distribution

debian 10

Terraform version

terraform -v
Terraform v0.12.15
+ provider.libvirt (unversioned)

Provider and libvirt versions

~/.terraform.d/plugins/terraform-provider-libvirt -version
/root/.terraform.d/plugins/terraform-provider-libvirt 0.6.0+git.1569597268.1c8597df
Compiled against library: libvirt 4.0.0
Using library: libvirt 5.0.0
Running hypervisor: QEMU 3.1.0
Running against daemon: 5.0.0

If that gives you "was not built correctly", get the Git commit hash from your local provider repository:

git describe --always --abbrev=40 --dirty

Checklist

[ ] Is your issue/contribution related with enabling some setting/option exposed by libvirt that the plugin does not yet support, or requires changing/extending the provider terraform schema?
- [ ] Make sure you explain why this option is important to you, why it should be important to everyone. Describe your use-case with detail and provide examples where possible.
- [ ] If it is a very special case, consider using the XSLT support in the provider to tweak the definition instead of opening an issue
- [ ] Maintainers do not have expertise in every libvirt setting, so please, describe the feature and how it is used. Link to the appropriate documentation
[x] Is it a bug or something that does not work as expected? Please make sure you fill the version information below:

Description of Issue/Question

Setup

provider "libvirt" {
    uri = "qemu:///system"
}

resource "libvirt_pool" "cluster" {
  name = "cluster"
  type = "dir"
  path = "/var/lib/libvirt/cluster"
}

resource "libvirt_volume" "debian10" {
  name   = "debian10"
  pool   = libvirt_pool.cluster.name
  source = "https://cdimage.debian.org/cdimage/openstack/current/debian-10.1.6-20191114-openstack-amd64.qcow2"
}

resource "libvirt_network" "cluster" {
  name = "cluster"
  addresses = ["192.168.23.0/24"]
}

resource "libvirt_volume" "cluster" {
  count = 3
  name = "cluster-${ count.index }"
  pool   = libvirt_pool.cluster.name
  base_volume_name = libvirt_volume.debian10.name
}

resource "libvirt_cloudinit_disk" "cloudinit" {
  name      = "cloudinit.iso"
  pool   = libvirt_pool.cluster.name
  user_data = <<EOF
#cloud-config
users:
- default
- name: cyril
  ssh_authorized_keys:
  - authkey
  sudo: true
EOF
}

resource "libvirt_domain" "cluster" {
  count = 3
  name = "cluster-${ count.index }"
  machine = "q35"
  disk {
    volume_id = libvirt_volume.cluster[count.index].id
    scsi = true
  }
  network_interface {
    network_id     = libvirt_network.cluster.id
  }
  console {
    type = "pty"
    target_port = "0"
  }
  cloudinit = libvirt_cloudinit_disk.cloudinit.id
}

Steps to Reproduce Issue


Error: Error defining libvirt domain: virError(Code=67, Domain=10, Message='unsupported configuration: IDE controllers are unsupported for this QEMU binary or machine type')

  on main.tf line 43, in resource "libvirt_domain" "cluster":
  43: resource "libvirt_domain" "cluster" {

Error: Error defining libvirt domain: virError(Code=67, Domain=10, Message='unsupported configuration: IDE controllers are unsupported for this QEMU binary or machine type')

  on main.tf line 43, in resource "libvirt_domain" "cluster":
  43: resource "libvirt_domain" "cluster" {

Error: Error defining libvirt domain: virError(Code=67, Domain=10, Message='unsupported configuration: IDE controllers are unsupported for this QEMU binary or machine type')

  on main.tf line 43, in resource "libvirt_domain" "cluster":
  43: resource "libvirt_domain" "cluster" {

Additional information:

Do you have SELinux or Apparmor/Firewall enabled? Some special configuration? Have you tried to reproduce the issue without them enabled?

MalloZup commented 4 years ago

@jouve thx for issue . I guess you have an perhaps outdated Libvirt/qemu or something that is out of scope of this provider.

'unsupported configuration: IDE controllers are unsupported for this QEMU binary or machine type' This is more related to libvirt/qemu of your host rather then provder error .

Or it could be also something related to the image you are using ( source = "https://cdimage.debian.org/cdimage/openstack/current/debian-10.1.6-20191114-openstack-amd64.qcow2"

let me know if an update/downgrade or soemthing in the libvirt/qemu conf pkg fixed it.

jouve commented 4 years ago

libvirt and qemu are less than 1 year old according to their release notes...

I made it work by mounting the cloud-init volume this way:

disk {
    volume_id = split(";", libvirt_cloudinit_disk.ceph[count.index].id)[0]
  }

MalloZup commented 4 years ago

we might need to look at what the error msg imply. Strange workaround :/ Especially with the split

If you can attach TF_LOG=DEBUG terraform apply with logs we might gain more clarity. :+1:

jouve commented 4 years ago

the split is there because the id is like /var/lib/libvirt/stack/ceph-0-cloudinit.iso;5dd93673-c0ce-4749-d92a-471989cd2108, so it retrieves the path to the cloudinit iso

I'll try the TF_LOG=DEBUG

jouve commented 4 years ago

this produce the error:

2019-11-23T21:08:43.482+0100 [DEBUG] plugin.terraform-provider-libvirt:       <devices>
2019-11-23T21:08:43.482+0100 [DEBUG] plugin.terraform-provider-libvirt:           <disk type="volume" device="disk">
2019-11-23T21:08:43.482+0100 [DEBUG] plugin.terraform-provider-libvirt:               <driver name="qemu" type="qcow2"></driver>
2019-11-23T21:08:43.482+0100 [DEBUG] plugin.terraform-provider-libvirt:               <source pool="k8s" volume="k8s-2.qcow2"></source>
2019-11-23T21:08:43.482+0100 [DEBUG] plugin.terraform-provider-libvirt:               <target dev="vda" bus="virtio"></target>
2019-11-23T21:08:43.482+0100 [DEBUG] plugin.terraform-provider-libvirt:           </disk>
2019-11-23T21:08:43.482+0100 [DEBUG] plugin.terraform-provider-libvirt:           <disk type="file" device="cdrom">
2019-11-23T21:08:43.482+0100 [DEBUG] plugin.terraform-provider-libvirt:               <driver name="qemu" type="raw"></driver>
2019-11-23T21:08:43.482+0100 [DEBUG] plugin.terraform-provider-libvirt:               <source file="/var/lib/libvirt/k8s/k8s-2-cloudinit.iso"></source>
2019-11-23T21:08:43.482+0100 [DEBUG] plugin.terraform-provider-libvirt:               <target dev="hdd" bus="ide"></target>
2019-11-23T21:08:43.482+0100 [DEBUG] plugin.terraform-provider-libvirt:           </disk>

this is ok:

2019-11-23T21:10:00.417+0100 [DEBUG] plugin.terraform-provider-libvirt:     <disk type='volume' device='disk'>
2019-11-23T21:10:00.417+0100 [DEBUG] plugin.terraform-provider-libvirt:       <driver name='qemu' type='qcow2'/>
2019-11-23T21:10:00.417+0100 [DEBUG] plugin.terraform-provider-libvirt:       <source pool='k8s' volume='k8s-0.qcow2'/>
2019-11-23T21:10:00.417+0100 [DEBUG] plugin.terraform-provider-libvirt:       <backingStore type='file' index='1'>
2019-11-23T21:10:00.417+0100 [DEBUG] plugin.terraform-provider-libvirt:         <format type='qcow2'/>
2019-11-23T21:10:00.417+0100 [DEBUG] plugin.terraform-provider-libvirt:         <source file='/var/lib/libvirt/k8s/debian-10-generic-amd64-20191113-76.qcow2'/>
2019-11-23T21:10:00.417+0100 [DEBUG] plugin.terraform-provider-libvirt:         <backingStore/>
2019-11-23T21:10:00.417+0100 [DEBUG] plugin.terraform-provider-libvirt:       </backingStore>
2019-11-23T21:10:00.417+0100 [DEBUG] plugin.terraform-provider-libvirt:       <target dev='vda' bus='virtio'/>
2019-11-23T21:10:00.417+0100 [DEBUG] plugin.terraform-provider-libvirt:       <alias name='virtio-disk0'/>
2019-11-23T21:10:00.417+0100 [DEBUG] plugin.terraform-provider-libvirt:       <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
2019-11-23T21:10:00.417+0100 [DEBUG] plugin.terraform-provider-libvirt:     </disk>
2019-11-23T21:10:00.417+0100 [DEBUG] plugin.terraform-provider-libvirt:     <disk type='volume' device='disk'>
2019-11-23T21:10:00.417+0100 [DEBUG] plugin.terraform-provider-libvirt:       <driver name='qemu' type='raw'/>
2019-11-23T21:10:00.417+0100 [DEBUG] plugin.terraform-provider-libvirt:       <source pool='k8s' volume='k8s-0-cloudinit.iso'/>
2019-11-23T21:10:00.417+0100 [DEBUG] plugin.terraform-provider-libvirt:       <backingStore/>
2019-11-23T21:10:00.417+0100 [DEBUG] plugin.terraform-provider-libvirt:       <target dev='vdb' bus='virtio'/>
2019-11-23T21:10:00.417+0100 [DEBUG] plugin.terraform-provider-libvirt:       <alias name='virtio-disk1'/>
2019-11-23T21:10:00.417+0100 [DEBUG] plugin.terraform-provider-libvirt:       <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
2019-11-23T21:10:00.417+0100 [DEBUG] plugin.terraform-provider-libvirt:     </disk>

jouve commented 4 years ago

@MalloZup I found out, the machine type q35 does not support IDE. SATA seems available for both q35 an i440FX, so maybe it would be a better choice ?

MalloZup commented 4 years ago

@jouve thx for investigation and effort. I think if the proposed change is generic and doesn't introduce any breaking points we can change it.

Can you provide a PR and also perhaps 2 links where we could take look of this machine types?

tia :sun_with_face:

vmorris commented 3 years ago

I also found this problem on s390x.

Generated xml:

<disk type="file" device="cdrom">
    <driver name="qemu" type="raw"></driver>
    <source file="/var/lib/libvirt/my_pool/cloud_init.iso"></source>
    <target dev="hdd" bus="ide"></target>
</disk>

This results in

2020/10/27 08:31:10 [DEBUG] libvirt_domain.server: apply errored, but we're indicating that via the Error pointer rather than returning it: Error defining libvirt domain: virError(Code=67, Domain=10, Message='unsupported configuration: IDE controllers are unsupported for this QEMU binary or machine type')

mattsn0w commented 3 years ago

I too am experiencing this issue. Machine type set to q35 seems to cause the error.

`2021-03-07T15:26:36.720-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021/03/07 15:26:36 [TRACE] Checking for x86_64/hvm against i686/hvm x86_64/hvm against x86_64/hvm 2021-03-07T15:26:36.720-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021/03/07 15:26:36 [DEBUG] Found 81 machines in guest for x86_64/hvm 2021-03-07T15:26:36.720-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021/03/07 15:26:36 [TRACE] Checking for x86_64/hvm against i686/hvm 2021-03-07T15:26:36.720-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021/03/07 15:26:36 [TRACE] Checking for x86_64/hvm against x86_64/hvm 2021-03-07T15:26:36.720-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021/03/07 15:26:36 [DEBUG] Found 81 machines in guest for x86_64/hvm 2021-03-07T15:26:36.726-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021/03/07 15:26:36 [DEBUG] Setting disk driver to 'qcow2' to match disk volume format 2021-03-07T15:26:36.726-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021/03/07 15:26:36 [DEBUG] scsiDisk: true 2021-03-07T15:26:36.726-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021/03/07 15:26:36 filesystems: [] 2021-03-07T15:26:36.728-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021/03/07 15:26:36 [INFO] Creating libvirt domain at qemu:///system 2021-03-07T15:26:36.729-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021/03/07 15:26:36 [DEBUG] Generated XML for libvirt domain: 2021-03-07T15:26:36.729-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.729-0700 [DEBUG] plugin.terraform-provider-libvirt: tfvm01 2021-03-07T15:26:36.729-0700 [DEBUG] plugin.terraform-provider-libvirt: 512 2021-03-07T15:26:36.729-0700 [DEBUG] plugin.terraform-provider-libvirt: 1 2021-03-07T15:26:36.729-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.729-0700 [DEBUG] plugin.terraform-provider-libvirt: hvm 2021-03-07T15:26:36.729-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.729-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.729-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.729-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.729-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.729-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.729-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.729-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: /usr/bin/qemu-system-x86_64 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 05abcdba8fbc8861 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: rtio-scsi"> 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: /dev/urandom 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021-03-07T15:26:36.730-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021/03/07 15:26:36 [DEBUG] libvirt_domain.tfvm01: apply errored, but we're indicating that via the Error pointer rather than returning it: Error defining libvirt domain: virError(Code=67, Domain=10, Message='unsupported configuration: IDE controllers are unsupported for this QEMU binary or machine type') 2021/03/07 15:26:36 [ERROR] eval: terraform.EvalApplyPost, err: Error defining libvirt domain: virError(Code=67, Domain=10, Message='unsupported configuration: IDE controllers are unsupported for this QEMU binary or machine type') 2021/03/07 15:26:36 [ERROR] eval: terraform.EvalSequence, err: Error defining libvirt domain: virError(Code=67, Domain=10, Message='unsupported configuration: IDE controllers are unsupported for this QEMU binary or machine type') 2021-03-07T15:26:36.841-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021/03/07 15:26:36 [INFO] Volume ID: /var/lib/libvirt/images/disk 2021-03-07T15:26:36.841-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021/03/07 15:26:36 Waiting for volume /var/lib/libvirt/images/disk to be active... 2021-03-07T15:26:36.841-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021/03/07 15:26:36 [DEBUG] Waiting for state to become: [EXISTS] 2021-03-07T15:26:36.843-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021/03/07 15:26:36 [DEBUG] Volume disk format: qcow2 2021-03-07T15:26:36.843-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021/03/07 15:26:36 [DEBUG] Unlocking "default" 2021-03-07T15:26:36.843-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021/03/07 15:26:36 [DEBUG] Unlocked "default" 2021/03/07 15:26:36 [WARN] Provider "registry.terraform.io/dmacvicar/libvirt" produced an unexpected new value for libvirt_volume.disk_ubuntu_resized, but we are tolerating it because it is using the legacy plugin SDK. The following problems may be the cause of any confusing errors from downstream operations:

.size: was cty.NumberIntVal(5.361393152e+09), but now cty.NumberIntVal(5.361393664e+09) libvirt_volume.disk_ubuntu_resized: Creation complete after 0s [id=/var/lib/libvirt/images/disk]

Error: Error defining libvirt domain: virError(Code=67, Domain=10, Message='unsupported configuration: IDE controllers are unsupported for this QEMU binary or machine type')

on ubuntu-focal-cluster.tf line 58, in resource "libvirt_domain" "tfvm01": 58: resource "libvirt_domain" "tfvm01" {

2021-03-07T15:26:36.853-0700 [DEBUG] plugin.terraform-provider-libvirt: 2021/03/07 15:26:36 [DEBUG] cleaning up connection for URI: qemu:///system 2021-03-07T15:26:36.856-0700 [DEBUG] plugin: plugin process exited: path=.terraform/plugins/registry.terraform.io/dmacvicar/libvirt/0.6.2/linux_amd64/terraform-provider-libvirt pid=2302085 2021-03-07T15:26:36.856-0700 [DEBUG] plugin: plugin exited msnow@stumpy:~/terraforms/ubuntu$ `

The-Loeki commented 3 years ago

hit by the same issue, thanks @jouve your workaround still works

dariush commented 3 years ago

Hit the same issue q35 might be a better 'machine choice in 2021 I would do some work and submit a PR, but my Go skills are non-existant :-(

In the meantime here's another workaround; an XSLT transform to convert the cdrom definition from IDE to SATA so it will work with a q35 machine. Hopefully it's useful to someone.

https://gist.github.com/dariush/7405cbf62835e03d0b5c953d798a87cd

There are side effects: Please understand that applying an xslt transform to the schema can have unpredictable effects. For example, if you alter the libvirt_cloudinit_disk resource definition after the xslt is applied and a domain (VM) is created and then run 'terraform apply' it will fail as the expected domain schema does not exist. You need to destroy and re-create the domain. In short YMMV :-)

scabala commented 5 days ago

Related to #1018

dmacvicar / terraform-provider-libvirt