search

dmacvicar / terraform-provider-libvirt

Terraform provider to provision infrastructure with Linux's KVM using libvirt
Apache License 2.0
1.6k stars 458 forks source link

Using Nic Mode Selector Causes Unexpected Side Effects #601

Open james-crowley opened 5 years ago

james-crowley commented 5 years ago

System Information

Linux distribution

Ubuntu 18.04

Terraform version

Terraform v0.11.13

Provider and libvirt versions

v0.5.1

Description of Issue/Question

Setup

Main TF File:

provider "libvirt" {
    uri = "qemu+ssh:TEMP"
}

resource "libvirt_domain" "terraform_test" {
  name = "terraform_test"
  memory = "4096"
  vcpu = 4
  network_interface {
    network_name = "ovswitch0"
  }

  xml {
    xslt = "${file("portgroup_select.xsl")}"
  }
}

XSL FILE:

<?xml version="1.0" ?>
<xsl:stylesheet version="1.0"
                xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output omit-xml-declaration="yes" indent="yes"/>
  <xsl:template match="node()|@*">
     <xsl:copy>
       <xsl:apply-templates select="node()|@*"/>
     </xsl:copy>
  </xsl:template>

  <xsl:template match="/domain/devices/interface[@type='network']/model/@type">
    <xsl:attribute name="type">
      <xsl:value-of select="'e1000'"/>
    </xsl:attribute>
  </xsl:template>

</xsl:stylesheet>

Steps to Reproduce Issue

I am trying to utilize XML option to due some edge case config and I noticed a side effect to using the NIC mode selector example.

Here is the XML dump(just network section) of the domain when I run the above TF without the XML option.

<domain type='kvm'>
  <devices>
    <interface type='network'>
      <mac address='42:13:67:b1:9f:3d'/>
      <source network='ovswitch0' portgroup='vlan-1284'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
  </devices>
</domain>

You'll notice that the source xml field is correct, it stats the correct network name and portgroup.

Now if apply the TF file with the NIC mode selector I get this:

<domain type='kvm' id='540'>
  <devices>
    <interface type='bridge'>
      <mac address='e6:d7:6a:3a:6e:f3'/>
      <source network='ovswitch0' bridge='vswitch0'/>
      <virtualport type='openvswitch'>
        <parameters interfaceid='0fd8e369-b087-459a-83cc-f0613d7d22f6'/>
      </virtualport>
      <target dev='vnet27'/>
      <model type='e1000'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
</domain>

The source field completely changed. It switched the network type to a bridge network when all I wanted to change was the model type from virtio to e1000.

Additional information:

There does not seem to be any related issues. There is issue #170 , which does have to do with Open VSwitches but I do not think they are running into the same issue.

james-crowley commented 5 years ago

So the above XML dump is from when the VM was running. Upon shutting it down and taking a XML dump I get:

<interface type='network'>
      <mac address='42:13:67:3d:3d:3d'/>
      <source network='ovswitch0'/>
      <model type='e1000'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>

Which almost correct, but you'll notice that the source does not have a the port group present. It looks like whenever I try to update any attribute in the interface block, it seems to drop the port group.

My end goal was to modify the port group before the VM starts up, since you do not have a terraform attribute for it. I saw you mentioned you will not be supporting all the little things to do with libvirt hence why you made the XML/XSL attribute. But there seems to be a bug with how its handling it.

james-crowley commented 5 years ago

So after some debugging I found out one of the issues. 

  <xsl:template match="/domain/devices/interface[@type='network']/source/@portgroup">
    <xsl:attribute name="portgroup">
      <xsl:value-of select="vlan-1284"/>
    </xsl:attribute>
  </xsl:template>

With the code above, I should be able to change the value of <source network='ovswitch0' portgroup='vlan-1'/> . The problem is when applying this config,

provider "libvirt" {
    uri = "qemu+ssh:/TEMP"
}

resource "libvirt_domain" "terraform_test" {
  name = "terraform_test"
  memory = "4096"
  vcpu = 4
  network_interface {
    network_name = "ovswitch0"
  }

  xml {
    xslt = "${file("portgroup_select.xsl")}"
  }
}

There is no option to declare a port group , which would be fine since I have my XSL file to take care of that. But after enabling debugging I saw this in the logs:

2019-05-29T17:12:59.529Z [DEBUG] plugin.terraform-provider-libvirt: 2019/05/29 17:12:59 [DEBUG] Generated XML for libvirt domain:
2019-05-29T17:12:59.529Z [DEBUG] plugin.terraform-provider-libvirt:   <domain type="kvm">
2019-05-29T17:12:59.529Z [DEBUG] plugin.terraform-provider-libvirt:       <name>terraform_test</name>
2019-05-29T17:12:59.529Z [DEBUG] plugin.terraform-provider-libvirt:       <memory unit="MiB">4096</memory>
2019-05-29T17:12:59.529Z [DEBUG] plugin.terraform-provider-libvirt:       <vcpu>5</vcpu>
2019-05-29T17:12:59.529Z [DEBUG] plugin.terraform-provider-libvirt:       <os>
2019-05-29T17:12:59.529Z [DEBUG] plugin.terraform-provider-libvirt:           <type>hvm</type>
2019-05-29T17:12:59.529Z [DEBUG] plugin.terraform-provider-libvirt:       </os>
2019-05-29T17:12:59.529Z [DEBUG] plugin.terraform-provider-libvirt:       <features>
2019-05-29T17:12:59.529Z [DEBUG] plugin.terraform-provider-libvirt:           <pae></pae>
2019-05-29T17:12:59.529Z [DEBUG] plugin.terraform-provider-libvirt:           <acpi></acpi>
2019-05-29T17:12:59.529Z [DEBUG] plugin.terraform-provider-libvirt:           <apic></apic>
2019-05-29T17:12:59.529Z [DEBUG] plugin.terraform-provider-libvirt:       </features>
2019-05-29T17:12:59.529Z [DEBUG] plugin.terraform-provider-libvirt:       <cpu></cpu>
2019-05-29T17:12:59.529Z [DEBUG] plugin.terraform-provider-libvirt:       <devices>
2019-05-29T17:12:59.530Z [DEBUG] plugin.terraform-provider-libvirt:           <interface type="network">
2019-05-29T17:12:59.530Z [DEBUG] plugin.terraform-provider-libvirt:               <mac address="22:75:21:69:c4:fd"></mac>
2019-05-29T17:12:59.530Z [DEBUG] plugin.terraform-provider-libvirt:               <source network="ovswitch0"></source>
2019-05-29T17:12:59.530Z [DEBUG] plugin.terraform-provider-libvirt:               <model type="virtio"></model>
2019-05-29T17:12:59.530Z [DEBUG] plugin.terraform-provider-libvirt:           </interface>
2019-05-29T17:12:59.530Z [DEBUG] plugin.terraform-provider-libvirt:           <channel>
2019-05-29T17:12:59.530Z [DEBUG] plugin.terraform-provider-libvirt:               <target type="virtio" name="org.qemu.guest_agent.0"></target>
2019-05-29T17:12:59.531Z [DEBUG] plugin.terraform-provider-libvirt:           </channel>
2019-05-29T17:12:59.531Z [DEBUG] plugin.terraform-provider-libvirt:           <graphics type="spice" autoport="yes"></graphics>
2019-05-29T17:12:59.531Z [DEBUG] plugin.terraform-provider-libvirt:           <rng model="virtio">
2019-05-29T17:12:59.531Z [DEBUG] plugin.terraform-provider-libvirt:               <backend model="random"></backend>
2019-05-29T17:12:59.531Z [DEBUG] plugin.terraform-provider-libvirt:           </rng>
2019-05-29T17:12:59.531Z [DEBUG] plugin.terraform-provider-libvirt:       </devices>
2019-05-29T17:12:59.531Z [DEBUG] plugin.terraform-provider-libvirt:   </domain>
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt: 2019/05/29 17:12:59 [DEBUG] Transformed XML with user specified XSLT:
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt: <domain type="kvm">
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt:       <name>terraform_test</name>
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt:       <memory unit="MiB">4096</memory>
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt:       <vcpu>5</vcpu>
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt:       <os>
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt:           <type>hvm</type>
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt:       </os>
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt:       <features>
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt:           <pae/>
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt:           <acpi/>
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt:           <apic/>
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt:       </features>
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt:       <cpu/>
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt:       <devices>
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt:           <interface type="network">
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt:               <mac address="22:75:21:69:c4:fd"/>
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt:               <source network="ovswitch0"/>
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt:               <model type="virtio"/>
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt:           </interface>
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt:           <channel>
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt:               <target type="virtio" name="org.qemu.guest_agent.0"/>
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt:           </channel>
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt:           <graphics type="spice" autoport="yes"/>
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt:           <rng model="virtio">
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt:               <backend model="random"/>
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt:           </rng>
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt:       </devices>
2019-05-29T17:12:59.539Z [DEBUG] plugin.terraform-provider-libvirt:   </domain>

You'll noticed on the generated XML for libvirt domain, when it comes to the interface block there is no mention of a port group.

With the XSL/XML is there away to inject an attribute instead of just modifying existing ones? If so, could you provide an example?

If adding the port group via XSL/XML is not support, could you consider adding it to the plugin as a terraform attribute?

james-crowley commented 5 years ago

This XML/XSL should work at assigning port groups.

<?xml version="1.0" ?>
<xsl:stylesheet version="1.0"
                xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output omit-xml-declaration="yes" indent="yes"/>
  <xsl:template match="node()|@*">
     <xsl:copy>
       <xsl:apply-templates select="node()|@*"/>
     </xsl:copy>
  </xsl:template>

  <xsl:template match="/domain/devices/interface[@type='network']/source/@*">
    <xsl:attribute name="network">
      <xsl:value-of select="'ovswitch1'"/>
    </xsl:attribute>

    <xsl:attribute name="portgroup">
      <xsl:value-of select="'vlan-1284'"/>
    </xsl:attribute>
  </xsl:template>
</xsl:stylesheet>

The @* makes every attribute in scope/accessible to that block.

Additionally, this XSL lets you set the network via the Terraform attribute and just changes the port group.

<?xml version="1.0" ?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output omit-xml-declaration="yes" indent="yes"/>
  <xsl:template match="node()|@*">
     <xsl:copy>
       <xsl:apply-templates select="node()|@*"/>
     </xsl:copy>
  </xsl:template>

  <xsl:template match="/domain/devices/interface[@type='network']/source/@*">
    <xsl:copy>
        <xsl:attribute name="network">
        </xsl:attribute>
    </xsl:copy>

    <xsl:attribute name="portgroup">
      <xsl:value-of select="'vlan-1284'"/>
    </xsl:attribute>
  </xsl:template>
</xsl:stylesheet>
MalloZup commented 5 years ago

thx @james-crowley for this. We might need to check it after the release 0.12 to terraform which we plan currenlty

james-crowley commented 5 years ago

@MalloZup All good. I was able to do the work around posted above to get it working, but would love to discuss the possibility of adding port groups to the plugin.

MalloZup commented 5 years ago

@james-crowley ok. Currenlty we might need to put it on the queue. If we get help from community for this from PR and PR is ok we can review it :+1: