node-public-ipv6 not having desired effect in IPv6 only cluster

dvgt commented 2 weeks ago

Expected Behavior

Setting the flannel.alpha.coreos.com/node-public-ipv6 annotation on a node should result in that IP address to be used as backend for the VXLAN tunnel.

Current Behavior

The IPv6 address that is getting used, is the first IPv6 address which is found on the interface that has the node-public-ipv6 address, which might not be the expected IPv6 address. Pod traffic going to other nodes are being sent using the first IPv6 address instead of the configured address from the annotation.

I think there is a small bug in match.go:LookupExtIface, called from main.go:282 or main.go:284. This passes the publicIP(v6) as argument to the function. When retrieving the interface for the given publicIP, the variable ifaceAddr is used initially https://github.com/flannel-io/flannel/blob/da774f2f523db109c3b9a81926de99c557ddbfca/pkg/ipmatch/match.go#L73 but when checking if a fallback is needed later, the ifaceV6Addr variable is used https://github.com/flannel-io/flannel/blob/da774f2f523db109c3b9a81926de99c557ddbfca/pkg/ipmatch/match.go#L243 which is not set in the case of an IPv6-only IP stack. Then the code falls back to using the first IPv6 address of the detected interface.

This does not happen in case of dual-stack because the ifaceV6Addr variable is set there https://github.com/flannel-io/flannel/blob/da774f2f523db109c3b9a81926de99c557ddbfca/pkg/ipmatch/match.go#L94

Possible Solution

A possible solution could be to add ifaceV6Addr = ifaceAddr and further use ifaceV6Addr above line https://github.com/flannel-io/flannel/blob/da774f2f523db109c3b9a81926de99c557ddbfca/pkg/ipmatch/match.go#L82 and potentially set ifaceAddr = nil to avoid processing it as an IPv4 address.

Steps to Reproduce (for bugs)

Setup an IPv6-only cluster (similar to https://github.com/flannel-io/flannel/issues/1813, but without the IPv4 addresses)
Add a second IPv6 address on the interface used for pod-to-pod communication on a certain node.
Set the flannel.alpha.coreos.com/node-public-ipv6 annotation to the original IPv6 address of the node.
Restart canal pods.

Context

Your Environment

Flannel version: rancher/hardened-flannel:v0.25.4-build20240610
Backend used: vxlan
Etcd version: rancher/hardened-etcd:v3.5.13-k3s1-build20240531
Kubernetes version (if used): v1.30.2+rke2r1
Operating System and version: Oracle Linux Server 8.10

rbrtbnfgl commented 1 week ago

Did you disable IPv4? On the net-conf configuration you should specify EnableIPv6: true and EnableIPv4: false

dvgt commented 1 day ago

This is the config map that's in use:

apiVersion: v1
kind: ConfigMap
metadata:
  annotations:
    meta.helm.sh/release-name: rke2-canal
    meta.helm.sh/release-namespace: kube-system
  creationTimestamp: "2024-09-18T12:15:44Z"
  labels:
    app.kubernetes.io/managed-by: Helm
  name: rke2-canal-config
  namespace: kube-system
data:
  canal_iface: ""
  canal_iface_regex: ""
  cni_network_config: |-
    {
      "name": "k8s-pod-network",
      "cniVersion": "0.3.1",
      "plugins": [
        {
          "type": "calico",
          "log_level": "info",
          "datastore_type": "kubernetes",
          "nodename": "__KUBERNETES_NODE_NAME__",
          "mtu": __CNI_MTU__,
          "ipam": {
              "type": "host-local",
              "ranges": [
                  [
                      {
                          "subnet": "usePodCidrIPv6"
                      }
                  ]
              ]
          },
          "policy": {
              "type": "k8s"
          },
          "kubernetes": {
              "kubeconfig": "__KUBECONFIG_FILEPATH__"
          }
        },
        {
          "type": "portmap",
          "snat": true,
          "capabilities": {"portMappings": true}
        },
        {
          "type": "bandwidth",
          "capabilities": {"bandwidth": true}
        }
      ]
    }
  masquerade: "true"
  net-conf.json: |
    {
      "EnableIPv4": false,
      "IPv6Network": "A:B:C:D::/108",
      "EnableIPv6": true,
      "Backend": {
        "Type": "vxlan"
      }
    }
  typha_service_name: none
  veth_mtu: "1450"

Note: The IPv6 address prefix was changed to A:B:C:D.

flannel-io / flannel