search

cloudbase / cloudbase-init

Cross-platform instance initialization
http://openstack.org
Apache License 2.0
428 stars 152 forks source link

VMware static IP metadata configuration #154

Open labserviceshslu opened 3 months ago

labserviceshslu commented 3 months ago

I am currently trying to get a working cloud-config metadata file for VMware to set a static IP. I am on v1.1.6 of cloudbase-init.

This is my metadata config:

#cloud-config
instance-id: test
local-hostname: test
admin-username: Admin
admin-password: Passw0rd
public-keys-data: ssh-ed25519 <key>
network:
  version: 2
  ethernets:
    Ethernet0:
      dhcp4: false
      addresses:
        - 10.0.0.10/24
      gateway4: 10.0.0.1
      nameservers:
        addresses:
          - 1.1.1.1
          - 1.0.0.1

My main question is about the "Ethernet0" part. In the example only matching the interface by MAC address is shown, however by cloud-init spec, matching the interface name directly (as shown above), matching the MAC and regex matching the name is supported.

Does cloudbase-init with the VMware metadata service only support matching by MAC address and would matching by name be something that could be added (if it isn't already)?

My use case is provisioning Windows VMs using terraform, so the metadata has to be generated/known at the same time as the VM is defined. I cannot wait for vSphere to assign a MAC address and then template the metadata after as metadata is written while the VM is created.

labserviceshslu commented 3 months ago

Just saw this issue which might resolve the problem described above: https://github.com/cloudbase/cloudbase-init/issues/134

However the example linked above does use version 2. And this line also looks like version 2 is supported.

These two PRs might be relevant as well (though I am not sure if the feature they want to add are already present?)

Edit: Just tried with version 1 config:

#cloud-config
instance-id: test
local-hostname: test
admin-username: Admin
admin-password: Passw0rd
public-keys-data: ssh-ed25519 <key>
network:
  version: 1
  config:
  - type: physical
    name: Ethernet0
    subnets:
      - type: static
        address: 10.0.0.10/24
        gateway: 10.0.0.1
        dns_nameservers: ["1.1.1.1","1.0.0.1"]

Version 1 or 2 does not seem to make a difference, I always get the same error message:

2024-08-12 22:29:37.768 3288 DEBUG cloudbaseinit.metadata.services.vmwareguestinfoservice [-] network data {'version': 1, 'config': [{'type': 'physical', 'name': 'Ethernet0', 'subnets': [{'type': 'static', 'address': '10.0.0.10/24', 'gateway': '10.0.0.1', 'dns_nameservers': ['1.1.1.1', '1.0.0.1']}]}]} _process_network_config C:\Program Files\Cloudbase Solutions\Cloudbase-Init\Python\Lib\site-packages\cloudbaseinit\metadata\services\vmwareguestinfoservice.py:240
2024-08-12 22:29:37.768 3288 DEBUG cloudbaseinit.utils.classloader [-] Loading class 'cloudbaseinit.osutils.windows.WindowsUtils' load_class C:\Program Files\Cloudbase Solutions\Cloudbase-Init\Python\Lib\site-packages\cloudbaseinit\utils\classloader.py:35
2024-08-12 22:29:37.768 3288 ERROR cloudbaseinit.init [-] plugin 'NetworkConfigPlugin' failed with error ''NoneType' object has no attribute 'lower''
2024-08-12 22:29:37.768 3288 ERROR cloudbaseinit.init [-] 'NoneType' object has no attribute 'lower': AttributeError: 'NoneType' object has no attribute 'lower'
2024-08-12 22:29:37.768 3288 ERROR cloudbaseinit.init Traceback (most recent call last):
2024-08-12 22:29:37.768 3288 ERROR cloudbaseinit.init   File "C:\Program Files\Cloudbase Solutions\Cloudbase-Init\Python\Lib\site-packages\cloudbaseinit\init.py", line 66, in _exec_plugin
2024-08-12 22:29:37.768 3288 ERROR cloudbaseinit.init     (status, reboot_required) = plugin.execute(service,
2024-08-12 22:29:37.768 3288 ERROR cloudbaseinit.init                                 ^^^^^^^^^^^^^^^^^^^^^^^
2024-08-12 22:29:37.768 3288 ERROR cloudbaseinit.init   File "C:\Program Files\Cloudbase Solutions\Cloudbase-Init\Python\Lib\site-packages\cloudbaseinit\plugins\common\networkconfig.py", line 307, in execute
2024-08-12 22:29:37.768 3288 ERROR cloudbaseinit.init     return self._process_network_details_v2(network_details)
2024-08-12 22:29:37.768 3288 ERROR cloudbaseinit.init            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-12 22:29:37.768 3288 ERROR cloudbaseinit.init   File "C:\Program Files\Cloudbase Solutions\Cloudbase-Init\Python\Lib\site-packages\cloudbaseinit\plugins\common\networkconfig.py", line 295, in _process_network_details_v2
2024-08-12 22:29:37.768 3288 ERROR cloudbaseinit.init     NetworkConfigPlugin._process_physical_links(
2024-08-12 22:29:37.768 3288 ERROR cloudbaseinit.init   File "C:\Program Files\Cloudbase Solutions\Cloudbase-Init\Python\Lib\site-packages\cloudbaseinit\plugins\common\networkconfig.py", line 197, in _process_physical_links
2024-08-12 22:29:37.768 3288 ERROR cloudbaseinit.init     adapter_name = osutils.get_network_adapter_name_by_mac_address(
2024-08-12 22:29:37.768 3288 ERROR cloudbaseinit.init                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-12 22:29:37.768 3288 ERROR cloudbaseinit.init   File "C:\Program Files\Cloudbase Solutions\Cloudbase-Init\Python\Lib\site-packages\cloudbaseinit\osutils\windows.py", line 804, in get_network_adapter_name_by_mac_address
2024-08-12 22:29:37.768 3288 ERROR cloudbaseinit.init     iface_index_list = [
2024-08-12 22:29:37.768 3288 ERROR cloudbaseinit.init                        ^
2024-08-12 22:29:37.768 3288 ERROR cloudbaseinit.init   File "C:\Program Files\Cloudbase Solutions\Cloudbase-Init\Python\Lib\site-packages\cloudbaseinit\osutils\windows.py", line 808, in <listcomp>
2024-08-12 22:29:37.768 3288 ERROR cloudbaseinit.init     net_addr["mac_address"].lower() == mac_address.lower()]
2024-08-12 22:29:37.768 3288 ERROR cloudbaseinit.init                                        ^^^^^^^^^^^^^^^^^
2024-08-12 22:29:37.768 3288 ERROR cloudbaseinit.init AttributeError: 'NoneType' object has no attribute 'lower'
2024-08-12 22:29:37.768 3288 ERROR cloudbaseinit.init

Relevant code: https://github.com/cloudbase/cloudbase-init/blob/7cb24ead71167dccb194a23fc69bb4916628b926/cloudbaseinit/plugins/common/networkconfig.py#L292-L313

FlorianLaunay commented 1 month ago

Hi @labserviceshslu Are you using the latest stable version ? Network config support was brought by f0fe66b in release 1.1.5

labserviceshslu commented 1 month ago

Hi @labserviceshslu Are you using the latest stable version ? Network config support was brought by f0fe66b in release 1.1.5

Yes, we were using the latest version at the time (should have been 1.1.6 then).

FlorianLaunay commented 1 month ago

My bad. I did not read all your error log.

It's not quite well documented but your network configuration must match a MAC address by using the match argument:

network:
  version: 2
  ethernets:
    Ethernet0:
      dhcp4: false
      match:
       macaddress: "xx:xx:xx:xx:xx:xx"
      addresses:
        - 10.0.0.10/24
      gateway4: 10.0.0.1
      nameservers:
        addresses:
          - 1.1.1.1
          - 1.0.0.1

Otherwise your configuration will not be applied

FlorianLaunay commented 1 month ago

According with @ader1990 said in this discussion : https://review.opendev.org/c/x/cloudbase-init/+/913734/5..8/cloudbaseinit/metadata/services/nocloudservice.py#b297

labserviceshslu commented 1 month ago

According with @ader1990 said in this discussion : https://review.opendev.org/c/x/cloudbase-init/+/913734/5..8/cloudbaseinit/metadata/services/nocloudservice.py#b297

Thank you for the reply. Unfortunately this is not possible in our scenario as the MAC address is generated at the same time the cloud-init data is generated (VMware vSphere + Terraform).

Is there any way around this or is adding an option without this limitation possible?

FlorianLaunay commented 1 month ago

Thank you for the reply. Unfortunately this is not possible in our scenario as the MAC address is generated at the same time the cloud-init data is generated (VMware vSphere + Terraform).

Is there any way around this or is adding an option without this limitation possible?

I'm also using Terraform with VMware vSphere and I solved this problem by generating static MAC adresses with random_integer and macaddress providers:

terraform {
  required_version = ">= 1.8.0"

  required_providers {
    vsphere = {
      source  = "hashicorp/vsphere"
      version = ">= 2.8.0"
    }

    random = {
      source = "hashicorp/random"
      version = ">= 3.6.1"
    }

    macaddress = {
      source = "ivoronin/macaddress"
      version = ">= 0.3.2"
    }
  }
}

resource "random_integer" "mac_prefixes" {
  # Generate random integer between 0 and 63 (00 and 3F in hexadecimal) for each network interface of each vm
  # Build a map to loop on with key corresponding to "vm_name.interface_name"
  for_each = tomap({
    for interface in flatten([
      for vm, vm_attr in local.virtual_machines : [
        for net, net_attr in vm_attr.networks : {
          vm_key  = vm
          net_key = net_attr.name
        }
      ]
    ]) : "${interface.vm_key}.${interface.net_key}" => interface
  })

  min = 0
  max = 63
}

resource "macaddress" "mac_addresses" {
  # Generate static mac address for each network interface of each vm
  # Build a map to loop on with key corresponding to "vm_name.interface_name"
  for_each = tomap({
    for interface in flatten([
      for vm, vm_attr in local.virtual_machines : [
        for net, net_attr in vm_attr.networks : {
          vm_key  = vm
          net_key = net_attr.name
        }
      ]
    ]) : "${interface.vm_key}.${interface.net_key}" => interface
  })

  # use 00:50:56 VMWare prefix followed by the random integer generated previously
  # https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-networking/GUID-F9243FED-F081-498F-B4A9-EF950292AF77.html#GUID-ADFECCE5-19E7-4A81-B706-171E279ACBCD__GUID-6A6CB8C9-B579-4AEA-867D-B5E448F65426
  prefix = [0, 80, 86, random_integer.mac_prefixes[each.key].result]

  lifecycle {
    # Ignore future changes on prefix
    ignore_changes = [
      prefix,
    ]
  }
}

In this example, the virtual machines list is provided by the virtual_machines local variable and this code is looping over it using for_each. Object key is corresponding to VM name and network interfaces are stored in a networks list of objects with a name attribute corresponding to interface name.