amazonlinux / amazon-linux-2023

Amazon Linux 2023
https://aws.amazon.com/linux/amazon-linux-2023/
Other
500 stars 38 forks source link

[Bug] - ip link show does not list Altname ENI ID and device number for instance launched with EKS AL2023 AMI #686

Closed thecloudfaq closed 2 months ago

thecloudfaq commented 2 months ago

Describe the bug In regular AL2023 Instance, ip link show the following additional alternative name e.g. altname enp0s5, altname eni-abc12345667890, altname device-number-0 format however instance launched with EKS AL2023 list only following alternative name e.g. altname enp0s5.

Is it possible to add the device-number-0 and eniid to the altname for the attached eni in the AL2023 EKS AMIs.

To Reproduce Steps to reproduce the behavior:

  1. Launch EC2 Instance with AMI ID e.g. amazon-eks-node-al2023-x86_64-standard-1.25-v.20240227 with one or more ENI attached
  2. Run command ip link show to view the altnames

Expected behavior In AL2023 OS , it show the following

  1. ens5: <BROADCAST .... ...... altname enp0s5 altname eni-0123456789 altname device-number-0 In EKS AL2023 OS , it only show the following
  2. ens5: <BROADCAST .... ...... altname enp0s5
elsaco commented 2 months ago

@thecloudfaq that information is defined by the AlternativeNamesPolicy setting. On latest AL2023 AMI (2023.4.20240401) is set to AlternativeNamesPolicy=database onboard slot path for the default link. Sample output on my test instance:

    altname eni-017c33ac0f38916f7
    altname device-number-0

See /usr/lib/systemd/network/99-default.link

What is the setting on the EKS image?

vigh-m commented 2 months ago

Those altnames are created by the amazon-ec2-net-utils package. I tested using the AMI ID: ami-0780414b9701470ff, amazon/amazon-eks-node-al2023-x86_64-standard-1.27-v20240213. While I can see the package is installed correctly when I try to trigger the udev rules which trigger the creation of the altnames I see the following

$ udevadm test /sys/class/net/ens5/
This program is for debugging only, it does not run any program
specified by a RUN key. It may show incorrect results, because
some values may be different, or not available at a simulation run.

Trying to open "/etc/systemd/hwdb/hwdb.bin"...
Trying to open "/etc/udev/hwdb.bin"...
=== trie on-disk ===
tool version:          252
file size:        12064583 bytes
header size             80 bytes
strings            2534895 bytes
nodes              9529608 bytes
Loading kernel module index.
Failed to read $container of PID 1, ignoring: Permission denied
Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy
Found container virtualization none.
Using default interface naming scheme 'v252'.
Parsed configuration file "/usr/lib/systemd/network/99-default.link"
Created link configuration context.
Skipping overridden file '/usr/lib/udev/rules.d/60-cdrom_id.rules'.
Skipping overridden file '/usr/lib/udev/rules.d/99-vpc-policy-routes.rules'.
Reading rules file: /usr/lib/udev/rules.d/01-md-raid-creating.rules
Reading rules file: /usr/lib/udev/rules.d/10-dm.rules
Reading rules file: /usr/lib/udev/rules.d/11-dm-lvm.rules
Reading rules file: /usr/lib/udev/rules.d/13-dm-disk.rules
Reading rules file: /usr/lib/udev/rules.d/50-udev-default.rules
Reading rules file: /usr/lib/udev/rules.d/51-ec2-hvm-devices.rules
Reading rules file: /usr/lib/udev/rules.d/51-ec2-xen-vbd-devices.rules
Reading rules file: /usr/lib/udev/rules.d/53-ec2-read-ahead-kb.rules
Reading rules file: /usr/lib/udev/rules.d/60-autosuspend.rules
Reading rules file: /usr/lib/udev/rules.d/60-block-scheduler.rules
[truncated for legibility]

Of note is the line Skipping overridden file '/usr/lib/udev/rules.d/99-vpc-policy-routes.rules'. That's the rule which would trigger amazon-ec2-net-utils to setup networking and generate altnames. My guess is during creation of the EKS AMI this rule was disabled in favor of something else.

I would reach out to EKS if possible to get more clarity on this. Hope this helps!

elsaco commented 2 months ago

If you look at the comments inside add_altnames() udev is not being used to add interface altnames:

    # Interface altnames can also be added using systemd .link files.
    # However, in order to use them, we need to wait until a
    # systemd-networkd reload operation completes and then trigger a
    # udev "move" event.  We avoid that overhead by adding the
    # altnames directly using ip(8).

Please see /usr/share/amazon-ec2-net-utils/lib.sh how AWS deals with altnames. The interesting function start at line 326.

stewartsmith commented 2 months ago

EKS uses an entirely different method to configure network interfaces. See https://github.com/aws/amazon-vpc-cni-k8s

As such, reach out to the EKS team over there as to this issue.