kubernetes / cloud-provider-aws

Cloud provider for AWS
https://cloud-provider-aws.sigs.k8s.io/
Apache License 2.0
374 stars 299 forks source link

DeviceIndex is not respected when processing multiple network interfaces on a node #911

Closed elmiko closed 1 month ago

elmiko commented 2 months ago

What happened:

In cases where a node has multiple network interfaces, the order in which they are listed from AWS is applied directly to the interfaces list that the CCM produces. Depending on the ordering from AWS, this may or may not be accurate for the node being deployed. Even when DeviceIndex is specified, it is not respected.

For example, given this data from AWS:

{
  "Reservations": [
    {
      "Instances": [
        {
          "NetworkInterfaces": [
            {
              "Attachment": {
                "DeviceIndex": 1,
              },
              "MacAddress": "02:0f:d9:6d:bf:eb",
              "PrivateIpAddress": "100.64.2.164",
            },
            {
              "Attachment": {
                "DeviceIndex": 0,
              },
              "MacAddress": "02:5c:60:51:b1:29",
              "PrivateIpAddress": "10.175.4.157",
            }
          ],
        }
      ],
    }
  ]
}

The interface with MAC 02:5c:60:51:b1:29 should be the first interface by its DeviceIndex, yet it is the second interface due to the ordering from AWS.

What you expected to happen:

When DeviceIndex is specified, I expect it to be honored when ordering the network interfaces.

How to reproduce it (as minimally and precisely as possible):

  1. create a node with multiple network interfaces, and device indexes specified
  2. ensure that the ordering from AWS is wrong from device index, or set the device indexes to be wrong based on the MAC address ordering
  3. inspect the primary network interface IP on the node object created

Anything else we need to know?:

this was solved in the in-tree cloud controller but appears to have not been ported to the out-of-tree controller, see https://github.com/kubernetes/kubernetes/pull/80747

this is the current code in the external ccm, https://github.com/openshift/cloud-provider-aws/blob/release-4.14/pkg/providers/v1/aws.go#L1515

Environment:

/kind bug

k8s-ci-robot commented 2 months ago

This issue is currently awaiting triage.

If cloud-provider-aws contributors determine this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
cartermckinnon commented 2 months ago

There's a PR pending for this: #909 👍

elmiko commented 2 months ago

great to hear, thanks @cartermckinnon !