OpenNebula / terraform-provider-opennebula

Terraform provider for OpenNebula
https://www.terraform.io/docs/providers/opennebula/
Mozilla Public License 2.0
61 stars 53 forks source link

VNet address reservation on VNet (inside cluster) looses cluster_ids information on subsequent terraform apply runs #556

Open PeterW-3E opened 3 weeks ago

PeterW-3E commented 3 weeks ago

Description

The vnet address reservation in a vnet (which is member of a OpenNebula cluster) is set up correctly in the first terraform apply run (incl. the correct cluster id assignment). On subsequent runs of terraform apply (without changing the input), the cluster id gets removed (and will remain removed in subsequent terraform apply runs).

This may create issues later when attaching NICs to the address reservation subnet (i.e. the NIC resource - VM, vRouter - would be on a different cluster than the address reservation subnet)

Terraform and Provider version

Terraform v1.8.5
on linux_amd64
+ provider registry.terraform.io/opennebula/opennebula v1.4.0

Affected resources and data sources

Terraform configuration

terraform {
  required_providers {
    opennebula = {
      source  = "OpenNebula/opennebula"
      version = "~> 1.4"
    }
  }
}

provider "opennebula" {
  endpoint = "https://x.x.x.x:p/RPC2"
}

######################################################################
# Note: the vNet already exists in OpenNebula as "NAT (Default)"
# and needs no creation by Terraform anymore.
# It is a 10.10.0.0/24 network (inside non-default cluster 100)
#
# We only require a reservation e.g. for the fixed IP assignment
# of a vRouter or VM

# retrieve the reference / information about that existing vNet:
data "opennebula_virtual_network" "private_vnet" {
  name = "NAT (Default)"
}

resource "opennebula_virtual_network_address_range" "private_vnet" {
  virtual_network_id = data.opennebula_virtual_network.private_vnet.id
  ar_type            = "IP4"
  size               = 1
  ip4                = "10.10.0.240"
}

resource "opennebula_virtual_network" "private_vnet_vrouter_ip" {
  name                 = "private_vnet_vrouter_ip"
  description          = "Address reservation vRouter NIC"
  reservation_vnet     = data.opennebula_virtual_network.private_vnet.id
  reservation_size     = 1
  reservation_ar_id    = opennebula_virtual_network_address_range.private_vnet.id
  reservation_first_ip = "10.10.0.240"
}

Expected behavior

On subsequent runs of terraform apply with unchanged terraform tf input the installed configuration should remain unchanged (i.e. the related cluster information should remain intact).

Actual behavior

Initialize terraform and create the address reservation subnet:

user@host:~$ terraform init

Initializing the backend...

Initializing provider plugins...
- Reusing previous version of opennebula/opennebula from the dependency lock file
- Using previously-installed opennebula/opennebula v1.4.0

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
user@host:~$ terraform show
The state file is empty. No resources are represented.
user@host:~$ terraform apply
data.opennebula_virtual_network.private_vnet: Reading...
data.opennebula_virtual_network.private_vnet: Read complete after 0s [name=NAT (Default)]

Terraform used the selected providers to generate the following execution plan. Resource
actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # opennebula_virtual_network.private_vnet_vrouter_ip will be created
  + resource "opennebula_virtual_network" "private_vnet_vrouter_ip" {
      + automatic_vlan_id    = (known after apply)
      + bridge               = (known after apply)
      + default_tags         = (known after apply)
      + description          = "Address reservation vRouter NIC"
      + gid                  = (known after apply)
      + gname                = (known after apply)
      + guest_mtu            = 1500
      + id                   = (known after apply)
      + mtu                  = 1500
      + name                 = "private_vnet_vrouter_ip"
      + permissions          = (known after apply)
      + physical_device      = (known after apply)
      + reservation_ar_id    = (known after apply)
      + reservation_first_ip = "10.10.0.240"
      + reservation_size     = 1
      + reservation_vnet     = 2
      + security_groups      = (known after apply)
      + tags_all             = (known after apply)
      + type                 = "bridge"
      + uid                  = (known after apply)
      + uname                = (known after apply)
      + vlan_id              = (known after apply)
    }

  # opennebula_virtual_network_address_range.private_vnet will be created
  + resource "opennebula_virtual_network_address_range" "private_vnet" {
      + ar_type            = "IP4"
      + held_ips           = (known after apply)
      + id                 = (known after apply)
      + ip4                = "10.10.0.240"
      + mac                = (known after apply)
      + size               = 1
      + virtual_network_id = 2
    }

Plan: 2 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

opennebula_virtual_network_address_range.private_vnet: Creating...
opennebula_virtual_network_address_range.private_vnet: Creation complete after 0s [id=6]
opennebula_virtual_network.private_vnet_vrouter_ip: Creating...
opennebula_virtual_network.private_vnet_vrouter_ip: Creation complete after 0s [id=92]

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.
user@host:~$ terraform show
# data.opennebula_virtual_network.private_vnet:
data "opennebula_virtual_network" "private_vnet" {
    id   = 2
    name = "NAT (Default)"
    tags = {
        "BRIDGE"              = "onebr1"
        "BRIDGE_TYPE"         = "linux"
        "DESCRIPTION"         = "Private NAT (Default) vNet. VLAN ID 1002."
        "DNS"                 = "9.9.9.9"
        "FILTER_IP_SPOOFING"  = "YES"
        "FILTER_MAC_SPOOFING" = "YES"
        "GATEWAY"             = "10.10.0.1"
        "NETWORK_ADDRESS"     = "10.10.0.0"
        "NETWORK_MASK"        = "255.255.255.0"
        "OUTER_VLAN_ID"       = null
        "PHYDEV"              = "enp16s0"
        "SECURITY_GROUPS"     = null
        "VLAN_ID"             = "1002"
        "VN_MAD"              = "802.1Q"
    }
}

# opennebula_virtual_network.private_vnet_vrouter_ip:
resource "opennebula_virtual_network" "private_vnet_vrouter_ip" {
    bridge               = "onebr1"
    cluster_ids          = [
        100,
    ]
    default_tags         = {}
    description          = "Address reservation vRouter NIC"
    gid                  = 0
    gname                = "oneadmin"
    guest_mtu            = 1500
    id                   = "92"
    mtu                  = 1500
    name                 = "private_vnet_vrouter_ip"
    permissions          = "600"
    physical_device      = "enp16s0"
    reservation_ar_id    = 6
    reservation_first_ip = "10.10.0.240"
    reservation_size     = 1
    reservation_vnet     = 2
    security_groups      = [
        0,
    ]
    tags_all             = {}
    type                 = "bridge"
    uid                  = 47
    uname                = "terraform"
    vlan_id              = "1002"
}

# opennebula_virtual_network_address_range.private_vnet:
resource "opennebula_virtual_network_address_range" "private_vnet" {
    ar_type            = "IP4"
    held_ips           = []
    id                 = "6"
    ip4                = "10.10.0.240"
    ip6                = null
    mac                = "02:00:0a:0a:00:f0"
    size               = 1
    ula_prefix         = null
    virtual_network_id = 2
}

So far, everything works as expected. Now rerun terraform apply with unchanged tf input file:

user@host:~$ terraform apply
data.opennebula_virtual_network.private_vnet: Reading...
data.opennebula_virtual_network.private_vnet: Read complete after 0s [name=NAT (Default)]
opennebula_virtual_network_address_range.private_vnet: Refreshing state... [id=6]
opennebula_virtual_network.private_vnet_vrouter_ip: Refreshing state... [id=92]

Terraform used the selected providers to generate the following execution plan. Resource
actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # opennebula_virtual_network.private_vnet_vrouter_ip will be updated in-place
  ~ resource "opennebula_virtual_network" "private_vnet_vrouter_ip" {
      ~ cluster_ids          = [
          - 100,
        ]
        id                   = "92"
        name                 = "private_vnet_vrouter_ip"
        # (19 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

opennebula_virtual_network.private_vnet_vrouter_ip: Modifying... [id=92]
opennebula_virtual_network.private_vnet_vrouter_ip: Modifications complete after 0s [id=92]

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

The cluster assignment has been removed from the reservation subnet.

user@host:~$ terraform show
# data.opennebula_virtual_network.private_vnet:
data "opennebula_virtual_network" "private_vnet" {
    id   = 2
    name = "NAT (Default)"
    tags = {
        "BRIDGE"              = "onebr1"
        "BRIDGE_TYPE"         = "linux"
        "DESCRIPTION"         = "Private NAT (Default) vNet. VLAN ID 1002."
        "DNS"                 = "9.9.9.9"
        "FILTER_IP_SPOOFING"  = "YES"
        "FILTER_MAC_SPOOFING" = "YES"
        "GATEWAY"             = "10.10.0.1"
        "NETWORK_ADDRESS"     = "10.10.0.0"
        "NETWORK_MASK"        = "255.255.255.0"
        "OUTER_VLAN_ID"       = null
        "PHYDEV"              = "enp16s0"
        "SECURITY_GROUPS"     = null
        "VLAN_ID"             = "1002"
        "VN_MAD"              = "802.1Q"
    }
}

# opennebula_virtual_network.private_vnet_vrouter_ip:
resource "opennebula_virtual_network" "private_vnet_vrouter_ip" {
    bridge               = "onebr1"
    cluster_ids          = []
    default_tags         = {}
    description          = "Address reservation vRouter NIC"
    gid                  = 0
    gname                = "oneadmin"
    guest_mtu            = 1500
    id                   = "92"
    mtu                  = 1500
    name                 = "private_vnet_vrouter_ip"
    permissions          = "600"
    physical_device      = "enp16s0"
    reservation_ar_id    = 6
    reservation_first_ip = "10.10.0.240"
    reservation_size     = 1
    reservation_vnet     = 2
    security_groups      = [
        0,
    ]
    tags_all             = {}
    type                 = "bridge"
    uid                  = 47
    uname                = "terraform"
    vlan_id              = "1002"
}

# opennebula_virtual_network_address_range.private_vnet:
resource "opennebula_virtual_network_address_range" "private_vnet" {
    ar_type            = "IP4"
    custom             = {}
    held_ips           = [
        "10.10.0.240",
    ]
    hold_ips           = []
    id                 = "6"
    ip4                = "10.10.0.240"
    ip6                = null
    mac                = "02:00:0a:0a:00:f0"
    size               = 1
    ula_prefix         = null
    virtual_network_id = 2
}

Additional reruns of terraform apply with still unchanged tf input don't change the situation:

user@host:~$ terraform apply
data.opennebula_virtual_network.private_vnet: Reading...
data.opennebula_virtual_network.private_vnet: Read complete after 0s [name=NAT (Default)]
opennebula_virtual_network_address_range.private_vnet: Refreshing state... [id=6]
opennebula_virtual_network.private_vnet_vrouter_ip: Refreshing state... [id=92]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no
differences, so no changes are needed.

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
user@host:~$ 

Steps to Reproduce

Debug output

No response

Panic output

No response

Important factoids

No response

References

No response