Error: Error defining libvirt network: virError(Code=3, Domain=19, Message='this function is not supported by the connection driver: virNetworkDefineXML')

prologic commented 5 years ago

System Information

Linux distribution

N/A

I am running macOS Mojave 10.14.5 along with Homebrew

$ brew --version
Homebrew 2.1.11-40-gd294881
Homebrew/homebrew-core (git revision b857; last commit 2019-09-09)
Homebrew/homebrew-cask (git revision b6582c; last commit 2019-09-09)

Terraform version

$ terraform -v
Terraform v0.12.8

Provider and libvirt versions

$ terraform-provider-libvirt -version
terraform-provider-libvirt was not built correctly
Compiled against library: libvirt 5.7.0
Using library: libvirt 5.7.0
Running hypervisor: QEMU 4.1.0
Running against daemon: 5.7.0

If that gives you "was not built correctly", get the Git commit hash from your local provider repository:

I did not install from Git -- I installed with go get ...

Checklist

[ ] Is your issue/contribution related with enabling some setting/option exposed by libvirt that the plugin does not yet support, or requires changing/extending the provider terraform schema?
- [ ] Make sure you explain why this option is important to you, why it should be important to everyone. Describe your use-case with detail and provide examples where possible.
- [ ] If it is a very special case, consider using the XSLT support in the provider to tweak the definition instead of opening an issue
- [ ] Maintainers do not have expertise in every libvirt setting, so please, describe the feature and how it is used. Link to the appropriate documentation
[ ] Is it a bug or something that does not work as expected? Please make sure you fill the version information below:

Description of Issue/Question

Setup

I am following this blog post and adapting it to my enviornment (macOS + libvirt).

Here are my files as they are laid out in ~/tmp/terraform:

cloud_init.cfg:

#cloud-config
users:
  - name: ubuntu
    sudo: ALL=(ALL) NOPASSWD:ALL
    groups: users, admin
    home: /home/ubuntu
    shell: /bin/bash
    ssh-authorized-keys:
      - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCu/v1OUkTW/Nc8ARP/TIwx5OenXmAovr5k9uzIQ9F/ntnk9m00w/kxy/+fMXZ5m1q4ZuBByO40vP8VtCIZTH8DBxzXRhQgnjoYhE2ug5Byyg+wAoZz74Kigi513/plwL9KU50giP/oaqlzK7YwI3qdiEgJ66gOjVIqhbKShVmB1HJiBSm0B/zVGDLgRsQQmceNeKKSEnswjIxSz52ZutC2EEjIYIxD8dE7IdFrgWlsZjKA1g0iUIsVM1sVQoO7ltKhzKfeSHXvd3cMkNDMmFCQcWTk96QWG1lgGDBOCjGJ0lMbm9eG6ZuHG/LzrVcWZQA7pTQunDwYQycQRz/kcpxP james@Jamess-MacBook-Pro.local
ssh_pwauth: True
disable_root: false
chpasswd:
  list: |
     ubuntu:linux
  expire: False
package_update: true
packages:
    - qemu-guest-agent
growpart:
  mode: auto
  devices: ['/']

main.tf:

provider "template" {
  version = "~> 2.1"
}

# instance the provider
provider "libvirt" {
  uri = "qemu:///session"
}

# We fetch the latest ubuntu release image from their mirrors
resource "libvirt_volume" "ubuntu-qcow2" {
  name = "ubuntu-qcow2"
  pool = "images"
  source = "https://cloud-images.ubuntu.com/releases/xenial/release/ubuntu-16.04-server-cloudimg-amd64-disk1.img"
  format = "qcow2"
}

# Create a network for our VMs
resource "libvirt_network" "vm_network" {
   name = "vm_network"
   addresses = ["10.0.1.0/24"]
   dhcp {
    enabled = true
   }
}

# Use CloudInit to add our ssh-key to the instance
resource "libvirt_cloudinit_disk" "commoninit" {
          name = "commoninit.iso"
          pool = "images"
          user_data = "${data.template_file.user_data.rendered}"
          network_config = "${data.template_file.network_config.rendered}"
        }

data "template_file" "user_data" {
  template = "${file("${path.module}/cloud_init.cfg")}"
}

data "template_file" "network_config" {
  template = "${file("${path.module}/network_config.cfg")}"
}

# Create the machine
resource "libvirt_domain" "domain-ubuntu" {
  name = "ubuntu-terraform"
  memory = "512"
  vcpu = 1

  cloudinit = "${libvirt_cloudinit_disk.commoninit.id}"

  network_interface {
    network_id = "${libvirt_network.vm_network.id}"
    network_name = "vm_network"
  }

  # IMPORTANT
  # Ubuntu can hang is a isa-serial is not present at boot time.
  # If you find your CPU 100% and never is available this is why
  console {
    type        = "pty"
    target_port = "0"
    target_type = "serial"
  }

  console {
      type        = "pty"
      target_type = "virtio"
      target_port = "1"
  }

  disk {
       volume_id = "${libvirt_volume.ubuntu-qcow2.id}"
  }
  graphics {
    type = "spice"
    listen_type = "address"
    autoport = "true"
  }
}

network_config.cfg:

version: 2
ethernets:
  ens3:
     dhcp4: true

NB: When initially installing libvirt with brew install libvirt I had to also create a volume to store images. The Brew formula for libvirt on macOS doesn't create any volumes by default.

images-pool.xml:

<pool type="dir">
  <name>images</name>
  <target>
    <path>/usr/local/var/lib/libvirt/images/</path>
  </target>
</pool>

Steps to Reproduce Issue

Create the images volume:

$ virsh pool-create images-pool.xml

Run terraform init

$ terraform init

Run terraform apply:

$ terraform apply

After some time creating the cloudinit images you get the following error:

libvirt_volume.ubuntu-qcow2: Still creating... [7m40s elapsed]
libvirt_cloudinit_disk.commoninit: Still creating... [7m40s elapsed]
libvirt_volume.ubuntu-qcow2: Creation complete after 7m48s [id=/usr/local/var/lib/libvirt/images/ubuntu-qcow2]
libvirt_cloudinit_disk.commoninit: Creation complete after 7m48s [id=/usr/local/var/lib/libvirt/images/commoninit.iso;5d75bb7e-c58c-172b-6aa6-ac8e6b56e04b]

Error: Error defining libvirt network: virError(Code=3, Domain=19, Message='this function is not supported by the connection driver: virNetworkDefineXML') -   <network>
      <name>vm_network</name>
      <forward mode="nat"></forward>
      <bridge stp="on"></bridge>
      <dns enable="no"></dns>
      <ip address="10.0.1.1" family="ipv4" prefix="24">
          <dhcp>
              <range start="10.0.1.2" end="10.0.1.254"></range>
          </dhcp>
      </ip>
  </network>

  on main.tf line 19, in resource "libvirt_network" "vm_network":
  19: resource "libvirt_network" "vm_network" {

Additional information:

Do you have SELinux or Apparmor/Firewall enabled? Some special configuration? Have you tried to reproduce the issue without them enabled?

No, no and no.

prologic commented 5 years ago

According to this (rather random unrelated find) unrelated issue it might be that libvirt isn't compiled with --with-virt-network -- So I'm giving that a go to see if that helps.

prologic commented 5 years ago

Re-installed the provider and added --with-virt-network to libvirt and built form source.

$ terraform-provider-libvirt --version
terraform-provider-libvirt fc4f30356f23f52f8a5766abea327e61d90e08e3-dirty
Compiled against library: libvirt 5.7.0
Using library: libvirt 5.7.0
Running hypervisor: QEMU 4.1.0
Running against daemon: 5.7.0

prologic commented 5 years ago

Probably helpful to know that virsh -V only reports "Remote Bridging" as a supported networking driver:

$ virsh -V
Virsh command line tool of libvirt 5.7.0
See web site at https://libvirt.org/

Compiled with support for:
 Hypervisors: QEMU/KVM VMware VirtualBox ESX Test
 Networking: Remote Bridging
 Storage: Dir
 Miscellaneous: Daemon Secrets Debug

wbogocki commented 5 years ago

I'm having the exact same issue, some relevant things:

Output of virsh -V:

Virsh command line tool of libvirt 5.7.0
See web site at https://libvirt.org/

Compiled with support for:
 Hypervisors: QEMU/KVM VMware VirtualBox ESX Test
 Networking: Remote Bridging
 Storage: Dir
 Miscellaneous: Daemon Secrets Debug

Output of ll /Users/stevejobs/.cache/libvirt/:

total 0
drwx------  2 stevejobs  staff    64B 30 Sep 17:30 hostdevmgr
drwxr-xr-x  5 stevejobs  staff   160B 30 Sep 19:09 qemu
drwx------  3 stevejobs  staff    96B 30 Sep 17:30 secrets
drwxr-xr-x  3 stevejobs  staff    96B 30 Sep 17:30 storage
srwx------  1 stevejobs  staff     0B  1 Oct 17:16 virtlogd-admin-sock
srwx------  1 stevejobs  staff     0B  1 Oct 17:16 virtlogd-sock

No virtnetworkd-sock file.

cclhsu commented 4 years ago

I have encountered the same issue. After some digging, I found that there is a description in libvirt configure .ac

dnl Make some notes about which OS we're compiling for, as the lxc and qemu
dnl drivers require linux headers, and storage_mpath, dtrace, and nwfilter
dnl are also linux specific.  The "network" and storage_fs drivers are known
dnl to not work on macOS presently, so we also make a note if compiling
dnl for that

scabala commented 2 months ago

Hi, does this still happen with latest version?

dmacvicar / terraform-provider-libvirt