Error creating virbr when having 2 networks

jlausuch commented 5 years ago

Trying to create a environment with 2 Networks, 2 VMs with 2 NICs each connected to the respective networks. This same environment, but with only 1 network doesn't give any problem.

System Information

Linux distribution

opensuse Leap 15.0

Terraform version

Terraform v0.11.13
+ provider.libvirt (unversioned)

Provider and libvirt versions

Compiled against library: libvirt 4.0.0
Using library: libvirt 4.0.0
Running hypervisor: QEMU 2.11.2
Running against daemon: 4.0.0

If that gives you "was not built correctly", get the Git commit hash from your local provider repository:

0.5.0+git.1542698041.34c84ded-lp150.1.1

Checklist

[ ] Is your issue/contribution related with enabling some setting/option exposed by libvirt that the plugin does not yet support, or requires changing/extending the provider terraform schema?
- [ ] Make sure you explain why this option is important to you, why it should be important to everyone. Describe your use-case with detail and provide examples where possible.
- [ ] If it is a very special case, consider using the XSLT support in the provider to tweak the definition instead of opening an issue
- [ ] Maintainers do not have expertise in every libvirt setting, so please, describe the feature and how it is used. Link to the appropriate documentation
[X] Is it a bug or something that does not work as expected? Please make sure you fill the version information below:

Description of Issue/Question

Setup

provider "libvirt" {
     uri = "qemu:///system"
}

resource "libvirt_volume" "myvdisk" {
  name = "my-vdisk-${count.index}.qcow2"
  pool = "default"
  count = 2
  source = "/var/lib/libvirt/images/my_image.qcow2"
  format = "qcow2"
}

resource "libvirt_network" "my_net1" {
   name = "my-net-1"
   addresses = ["10.10.0.0/24"]
   dhcp {
        enabled = true
   }
}

resource "libvirt_network" "my_net2" {
   name = "my-net-2"
   addresses = ["10.20.0.0/24"]
   dhcp {
        enabled = true
   }
}

resource "libvirt_domain" "domain-sle" {
  name = "test-vm-${count.index}"
  memory = "1024"
  vcpu = 1
  count = 2

  network_interface {
    network_id = "${libvirt_network.my_net1.id}"
    wait_for_lease = true
  }

  network_interface {
    network_id = "${libvirt_network.my_net2.id}"
    wait_for_lease = true
  }

  disk {
   volume_id = "${libvirt_volume.myvdisk.*.id[count.index]}"
  }

  console {
      type        = "pty"
      target_type = "virtio"
      target_port = "1"
  }
}

Steps to Reproduce Issue

+ terraform init

Initializing provider plugins...

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

+ ip a
<removed eth0 info>
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever

+ terraform apply -auto-approve
libvirt_volume.myvdisk[1]: Creating...
  format: "" => "qcow2"
  name:   "" => "my-vdisk-1.qcow2"
  pool:   "" => "default"
  size:   "" => "<computed>"
  source: "" => "/var/lib/libvirt/images/my_image.qcow2"
libvirt_volume.myvdisk[0]: Creating...
  format: "" => "qcow2"
  name:   "" => "my-vdisk-0.qcow2"
  pool:   "" => "default"
  size:   "" => "<computed>"
  source: "" => "/var/lib/libvirt/images/my_image.qcow2"
libvirt_network.my_net2: Creating...
  addresses.#:    "" => "1"
  addresses.0:    "" => "10.20.0.0/24"
  bridge:         "" => "<computed>"
  dhcp.#:         "" => "1"
  dhcp.0.enabled: "" => "true"
  mode:           "" => "nat"
  name:           "" => "my-net-2"
libvirt_network.my_net1: Creating...
  addresses.#:    "" => "1"
  addresses.0:    "" => "10.10.0.0/24"
  bridge:         "" => "<computed>"
  dhcp.#:         "" => "1"
  dhcp.0.enabled: "" => "true"
  mode:           "" => "nat"
  name:           "" => "my-net-1"
libvirt_volume.myvdisk[1]: Creation complete after 4s (ID: /var/lib/libvirt/images/my-vdisk-1.qcow2)
libvirt_network.my_net2: Creation complete after 5s (ID: fb13ec53-1849-4e5b-ac77-5c4730213abe)
libvirt_volume.myvdisk[0]: Creation complete after 9s (ID: /var/lib/libvirt/images/my-vdisk-0.qcow2)

Error: Error applying plan:

1 error(s) occurred:

* libvirt_network.my_net1: 1 error(s) occurred:

* libvirt_network.my_net1: Error clearing libvirt network: virError(Code=38, Domain=0, Message='error creating bridge interface virbr1: File exists')

Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.

+ ip a
<removed eth0 info>
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
439: virbr1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    link/ether 52:54:00:23:c5:46 brd ff:ff:ff:ff:ff:ff
    inet 10.20.0.1/24 brd 10.20.0.255 scope global virbr1
       valid_lft forever preferred_lft forever
440: virbr1-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr1 state DOWN group default qlen 1000
    link/ether 52:54:00:23:c5:46 brd ff:ff:ff:ff:ff:ff

jlausuch commented 5 years ago

This can be solved by hard-coding bridge names bridge = "my-bridge-1" and bridge = "my-bridge-2" in the network resources. I guess the expected behavior is that he framework calculates the bridge names automatically, and no need to specify it manually... So, I keep this issue open.

p0tr3c commented 4 years ago

There is a race condition between multiple network resources. Another workaround is to use depends_on between resources. I believe the issue is with the code looking up next available bridge name, and multiple resources fetch the same bridge name in parallel.

Is there any proffered method to implement the locking mechanism? I can spend some time to try to fix this, but don't want to waste my time if there is already a standard way of implementing locking for providers?

MalloZup commented 4 years ago

@p0tr3c https://github.com/dmacvicar/terraform-provider-libvirt/blob/c573bc15f8db231415b150d295e0063465e96735/libvirt/resource_libvirt_pool.go#L90 https://github.com/dmacvicar/terraform-provider-libvirt/blob/b7dab08a9194e388a7f043a1ab7fe60d241b50e5/libvirt/config.go#L17

and https://github.com/dmacvicar/terraform-provider-libvirt/blob/b7dab08a9194e388a7f043a1ab7fe60d241b50e5/libvirt/config.go#L4

hth

dmacvicar / terraform-provider-libvirt