search

canonical / cloud-init

Official upstream for the cloud-init: cloud instance initialization
https://cloud-init.io/
Other
2.86k stars 855 forks source link

NoCloud - network_config bridges incorrectly configured by sysconfig #5574

Open xiachen-rh opened 1 month ago

xiachen-rh commented 1 month ago

Bug report

Creating two bridges on top of two interfaces using Network-Config version 2 for cloud-init, but it seems that sysconfig creates "duplicated" ifcfg-* files and the bridge status is down. https://issues.redhat.com/browse/RHEL-38927

Environment details

How reproducible: 100%

Steps to reproduce the problem

Create a config drive ISO file with files: meta-data instance-id: local-hostname:

user-data

#cloud-config
users:
- name: kni
  ssh_authorized_keys:
  - <% INSERT SSH PUB KEY %>
  sudo: ALL=(ALL) NOPASSWD:ALL

network-config

 network:
  bridges:
    baremetal:
      addresses:
      - fc00:1:1::2/64
      interfaces:
      - baremetalport
      nameservers:
        addresses:
        - fc00:1:1::1
      routes:
      - on-link: true
        to: ::/0
        via: fc00:1:1::1
      stp: false
    provisioning:
      addresses:
      - fc00:1:2::2/64
      interfaces:
      - provisioningport
      routes: []
      stp: false
  ethernets:
    baremetalport:
      match:
        macaddress: 52:54:00:bd:8f:cb
      set-name: baremetal0
    provisioningport:
      match:
        macaddress: 52:54:00:25:ae:12
      set-name: provisioning0
  renderer: NetworkManager
  version: 2

Command used to build config-drive ISO image: genisoimage -output config-drive.iso -volid CIDATA -joliet \ -rock user-data meta-data \ network-config

Create a VM in libvirt with two network interfaces, match MAC address of network-config. Boot the VM

Expected results The VM should boot and correctly configure the network interfaces and bridges.

Actual results In total 6 ifcfg-* file were created, but existence of these four files seems suspicious: /etc/sysconfig/network-scripts/ifcfg-provisioning0 /etc/sysconfig/network-scripts/ifcfg-provisioningport /etc/sysconfig/network-scripts/ifcfg-baremetal0 /etc/sysconfig/network-scripts/ifcfg-baremetalport Is set-name not being processed correctly by sysconfig renderer?

$ nmcli -f NAME,FILENAME c show NAME FILENAME
Bridge provisioning /etc/sysconfig/network-scripts/ifcfg-provisioning
System provisioning0 /etc/sysconfig/network-scripts/ifcfg-provisioning0
System provisioningport /etc/sysconfig/network-scripts/ifcfg-provisioningport
Bridge baremetal /etc/sysconfig/network-scripts/ifcfg-baremetal
lo /run/NetworkManager/system-connections/lo.nmconnection System baremetal0 /etc/sysconfig/network-scripts/ifcfg-baremetal0
System baremetalport /etc/sysconfig/network-scripts/ifcfg-baremetalport 

provisioner login: kni 
Password: 
[kni@provisioner ~]$ sudo su -
Last failed login: Tue May 28 04:57:03 EDT 2024 on ttyS0
There were 3 failed login attempts since the last successful login.
[root@provisioner ~]# for file in $(find /etc/sysconfig/network-scripts/ -type f -name "ifcfg-*"); do echo "#### FILE: $file"; cat $file; done
#### FILE: /etc/sysconfig/network-scripts/ifcfg-baremetalport
# Created by cloud-init automatically, do not edit.
#
AUTOCONNECT_PRIORITY=120
BOOTPROTO=none
BRIDGE=baremetal
DEVICE=baremetalport
ONBOOT=yes
TYPE=Ethernet
USERCTL=no
#### FILE: /etc/sysconfig/network-scripts/ifcfg-baremetal
# Created by cloud-init automatically, do not edit.
#
AUTOCONNECT_PRIORITY=120
BOOTPROTO=none
DEFROUTE=yes
DEVICE=baremetal
DNS1=fc00:1:1::1
IPV6ADDR=fc00:1:1::2/64
IPV6INIT=yes
IPV6_AUTOCONF=no
IPV6_DEFAULTGW=fc00:1:1::1
IPV6_FORCE_ACCEPT_RA=no
ONBOOT=yes
TYPE=Bridge
USERCTL=no
#### FILE: /etc/sysconfig/network-scripts/ifcfg-provisioningport
# Created by cloud-init automatically, do not edit.
#
AUTOCONNECT_PRIORITY=120
BOOTPROTO=none
BRIDGE=provisioning
DEVICE=provisioningport
ONBOOT=yes
TYPE=Ethernet
USERCTL=no
#### FILE: /etc/sysconfig/network-scripts/ifcfg-provisioning
# Created by cloud-init automatically, do not edit.
#
AUTOCONNECT_PRIORITY=120
BOOTPROTO=none
DEVICE=provisioning
IPV6ADDR=fc00:1:2::2/64
IPV6INIT=yes
IPV6_AUTOCONF=no
IPV6_FORCE_ACCEPT_RA=no
ONBOOT=yes
TYPE=Bridge
USERCTL=no
#### FILE: /etc/sysconfig/network-scripts/ifcfg-baremetal0
# Created by cloud-init automatically, do not edit.
#
AUTOCONNECT_PRIORITY=120
BOOTPROTO=none
DEVICE=baremetal0
HWADDR=52:54:00:bd:8f:cb
ONBOOT=yes
TYPE=Ethernet
USERCTL=no
#### FILE: /etc/sysconfig/network-scripts/ifcfg-provisioning0
# Created by cloud-init automatically, do not edit.
#
AUTOCONNECT_PRIORITY=120
BOOTPROTO=none
DEVICE=provisioning0
HWADDR=52:54:00:25:ae:12
ONBOOT=yes
TYPE=Ethernet
USERCTL=no

[root@provisioner ~]# nmcli 
provisioning: connected to Bridge provisioning
        "provisioning"
        bridge, 52:54:00:BD:8F:CB, sw, mtu 1500
        inet6 fe80::d433:99ff:fec0:ad34/64
        inet6 fc00:1:2::2/64
        route6 fc00:1:2::/64 metric 425
        route6 fe80::/64 metric 1024provisioning0: connected to System provisioning0
        "Red Hat Virtio"
        ethernet (virtio_net), 52:54:00:25:AE:12, hw, mtu 1500
        inet6 fe80::5054:ff:fe25:ae12/64
        route6 fe80::/64 metric 256baremetal0: connected to System provisioningport
        "Red Hat Virtio"
        ethernet (virtio_net), 52:54:00:BD:8F:CB, hw, mtu 1500
        master provisioningbaremetal: connecting (getting IP configuration) to Bridge baremetal
        "baremetal"
        bridge, 92:3E:67:F1:40:6F, sw, mtu 1500
        ip6 defaultlo: connected (externally) to lo
        "lo"
        loopback (unknown), 00:00:00:00:00:00, sw, mtu 65536
        inet4 127.0.0.1/8
        inet6 ::1/128
        route6 ::1/128 metric 256DNS configuration:
        servers: fc00:1:1::1
        interface: baremetalUse "nmcli device show" to get complete information about known devices and
"nmcli connection show" to get an overview on active connection profiles.Consult nmcli(1) and nmcli-examples(7) manual pages for complete usage details.

cloud-init log

cloud-init.log

xiachen-rh commented 1 month ago

debug

I modified the network-config and set the value of interfaces under bridges same as "set-name" under ethernets , then the "duplicated" connections were gone and the bridges were up successfully. 

network:
  bridges:
    baremetal:
      addresses:
      - fc00:1:1::2/64
      interfaces:
      - baremetal0
      nameservers:
        addresses:
        - fc00:1:1::1
      routes:
      - on-link: true
        to: ::/0
        via: fc00:1:1::1
    provisioning:
      addresses:
      - fc00:1:2::2/64
      interfaces:
      - provisioning0
      routes: []
  ethernets:
    baremetalport:
      match:
        macaddress: 52:54:00:bd:8f:cb
      set-name: baremetal0
    provisioningport:
      match:
        macaddress: 52:54:00:25:ae:12
      set-name: provisioning0
  renderer: NetworkManager
  version: 2

the result details of network configuration

[kni@provisioner ~]$ ls /sys/class/net/
baremetal  baremetal0  lo  provisioning  provisioning0
[kni@provisioner ~]$ sudo su -
[root@provisioner ~]# for file in $(find /etc/sysconfig/network-scripts/ -type f -name "ifcfg-*"); do echo "#### FILE: $file"; cat $file; done
#### FILE: /etc/sysconfig/network-scripts/ifcfg-baremetal0
# Created by cloud-init automatically, do not edit.
#
AUTOCONNECT_PRIORITY=120
BOOTPROTO=none
BRIDGE=baremetal
DEVICE=baremetal0
HWADDR=52:54:00:bd:8f:cb
ONBOOT=yes
TYPE=Ethernet
USERCTL=no
#### FILE: /etc/sysconfig/network-scripts/ifcfg-baremetal
# Created by cloud-init automatically, do not edit.
#
AUTOCONNECT_PRIORITY=120
BOOTPROTO=none
DEFROUTE=yes
DEVICE=baremetal
DNS1=fc00:1:1::1
IPV6ADDR=fc00:1:1::2/64
IPV6INIT=yes
IPV6_AUTOCONF=no
IPV6_DEFAULTGW=fc00:1:1::1
IPV6_FORCE_ACCEPT_RA=no
ONBOOT=yes
TYPE=Bridge
USERCTL=no
#### FILE: /etc/sysconfig/network-scripts/ifcfg-provisioning0
# Created by cloud-init automatically, do not edit.
#
AUTOCONNECT_PRIORITY=120
BOOTPROTO=none
BRIDGE=provisioning
DEVICE=provisioning0
HWADDR=52:54:00:25:ae:12
ONBOOT=yes
TYPE=Ethernet
USERCTL=no
#### FILE: /etc/sysconfig/network-scripts/ifcfg-provisioning
# Created by cloud-init automatically, do not edit.
#
AUTOCONNECT_PRIORITY=120
BOOTPROTO=none
DEVICE=provisioning
IPV6ADDR=fc00:1:2::2/64
IPV6INIT=yes
IPV6_AUTOCONF=no
IPV6_FORCE_ACCEPT_RA=no
ONBOOT=yes
TYPE=Bridge
USERCTL=no

I reviewed the cloud-init document https://cloudinit.readthedocs.io/en/latest/reference/network-config-format-v2.html, it does not provide an example of interfaces under bridges and set-name under ethernets, so is it an issue of cloud-init or a configuration data problem?

ani-sinha commented 1 month ago

cc: @TheRealFalcon @holmanb @blackboxsw

ani-sinha commented 3 weeks ago

Ping on this ...

TheRealFalcon commented 2 weeks ago

so is it an issue of cloud-init or a configuration data problem?

This is an issue. Cloud-init should work as you expect given the first configuration.

ani-sinha commented 2 weeks ago

so is it an issue of cloud-init or a configuration data problem?

This is an issue. Cloud-init should work as you expect given the first configuration.

OK can we have a fix @TheRealFalcon ?

TheRealFalcon commented 2 weeks ago

Yes, though I can't make any guarantees about timelines.