[BUG] Error in subnet calculation of usingIPs

[Longchuanzheng]

Kube-OVN Version

v1.12-mc

Kubernetes Version

v1.21.1

Operation-system/Kernel Version

4.19

Description

Error in subnet calculation of usingIPs. While I check ip cr and ovn-nbctl show, I find nothing about this ip.

Steps To Reproduce

It's only happend on one subnet

Current Behavior

Error in subnet calculation of usingIPs.

Expected Behavior

Subnet calculation of usingIPs right.

[dosubot[bot]]

I found some similar closed issues that might be relevant to your problem:

1. subnet status v4usingIPrange not stay the same with v4usingIPs: This issue was closed, and the suggested solution was that the v4usingIPs count was correct, but there were some residual IPs in v4usingIPrange due to OVN EIPs ^[1].

2. ip conflict: This issue was also closed, and the suggested solution was to upgrade the version and observe if the issue persists ^[2].

3. error syncing 'master2': ovn-nbctl: Error on switch join: duplicate IPv4 address '100.64.0.4' found on logical switch port 'node-master3': This issue was closed without a specific solution provided ^[3].

To resolve the error in subnet calculation of usingIPs in Kube-OVN v1.12-mc on Kubernetes v1.21.1 with a 4.19 kernel, you need to ensure that the calculation of usingIPs correctly accounts for all IPs in use, including virtual IPs (vips), iptables EIPs, and OVN EIPs. The relevant code snippet for this calculation is:

// gateway always in excludeIPs
toSubIPs := util.ExpandExcludeIPs(subnet.Spec.ExcludeIps, subnet.Spec.CIDRBlock)
availableIPs := util.AddressCount(cidr) - util.CountIPNums(toSubIPs)
usingIPs := float64(usingIPNums)
vips, err := c.virtualIpsLister.List(labels.SelectorFromSet(labels.Set{
    util.SubnetNameLabel: subnet.Name,
    util.IPReservedLabel: "",
}))
if err != nil {
    klog.Error(err)
    return nil, err
}
lenVip = len(vips)
usingIPs += float64(lenVip)
if !isOvnSubnet(subnet) {
    eips, err := c.iptablesEipsLister.List(
        labels.SelectorFromSet(labels.Set{util.SubnetNameLabel: subnet.Name}))
    if err != nil {
        klog.Error(err)
        return nil, err
    }
    lenIptablesEip = len(eips)
    usingIPs += float64(lenIptablesEip)
}
if subnet.Spec.Vlan != "" {
    ovnEips, err := c.ovnEipsLister.List(labels.SelectorFromSet(labels.Set{
        util.SubnetNameLabel: subnet.Name,
    }))
    if err != nil {
        klog.Error(err)
        return nil, err
    }
    lenOvnEip = len(ovnEips)
    usingIPs += float64(lenOvnEip)
}

availableIPs -= usingIPs
if availableIPs < 0 {
    availableIPs = 0
}

v4UsingIPStr, v6UsingIPStr, v4AvailableIPStr, v6AvailableIPStr := c.ipam.GetSubnetIPRangeString(subnet.Name, subnet.Spec.ExcludeIps)
cachedFloatFields := [4]float64{
    subnet.Status.V4AvailableIPs,
    subnet.Status.V4UsingIPs,
    subnet.Status.V6AvailableIPs,
    subnet.Status.V6UsingIPs,
}
cachedStringFields := [4]string{
    subnet.Status.V4UsingIPRange,
    subnet.Status.V4AvailableIPRange,
    subnet.Status.V6UsingIPRange,
    subnet.Status.V6AvailableIPRange,
}
if subnet.Spec.Protocol == kubeovnv1.ProtocolIPv4 {
    subnet.Status.V4AvailableIPs = availableIPs
    subnet.Status.V4UsingIPs = usingIPs
    subnet.Status.V4UsingIPRange = v4UsingIPStr
    subnet.Status.V4AvailableIPRange = v4AvailableIPStr
    subnet.Status.V6AvailableIPs = 0
    subnet.Status.V6UsingIPs = 0
} else {
    subnet.Status.V6AvailableIPs = availableIPs
    subnet.Status.V6UsingIPs = usingIPs
    subnet.Status.V6UsingIPRange = v6UsingIPStr
    subnet.Status.V6AvailableIPRange = v6AvailableIPStr
    subnet.Status.V4AvailableIPs = 0
    subnet.Status.V4UsingIPs = 0
}
if cachedFloatFields == [4]float64{
    subnet.Status.V4AvailableIPs,
    subnet.Status.V4UsingIPs,
    subnet.Status.V6AvailableIPs,
    subnet.Status.V6UsingIPs,
} && cachedStringFields == [4]string{
    subnet.Status.V4UsingIPRange,
    subnet.Status.V4AvailableIPRange,
    subnet.Status.V6UsingIPRange,
    subnet.Status.V6AvailableIPRange,
} {
    return subnet, nil
}

bytes, err := subnet.Status.Bytes()
if err != nil {
    klog.Error(err)
    return nil, err
}
newSubnet, err := c.config.KubeOvnClient.KubeovnV1().Subnets().Patch(context.Background(), subnet.Name, types.MergePatchType, bytes, metav1.PatchOptions{}, "status")
return newSubnet, err

Ensure that the usingIPs calculation includes all relevant IPs and that the availableIPs is correctly adjusted. If there are any discrepancies, they should be addressed by verifying the logic and data sources used in the calculation ^[4].

To continue talking to Dosu, mention @dosu.