use tcnl.Class().Add(), but received "netlink receive: invalid argument"

[SMALL-head]

Describe the issue

I am learning TC recently. I want to do something like $TC class add dev {ifName} parent {parent} classid {classid} htb rate {limit}, but received netlink receive: invalid argument as a result.

Minimal code example to reproduce the issue

Because I don't want to change the network config in host, I call AddHTBClass in Linux network namespace(ip nets add ns1, BTW, I'm using Ubuntu20.04). I noticed that tc.Config has NetNS attribute, but I'm not sure how to deal with it, so I use another way to make the func works in NS correctly. To make the ns environment, use the bash below:

ip netns add ns1
ovs-vsctl add-br br0
ifconfig br0 up
ip link add v-ns1 type veth peer name b-ns1
ovs-vsctl add-port br0 b-ns1
ifconfig b-ns1 up
ip link set v-ns1 netns ns1
ip netns exec ns1 ifconfig v-ns1 up

Here's my main func. in this function, v-ns1 is one side of a veth pair where the other side b-ns1 is set on an ovs in the host .

package main

import (
        "net"

    "github.com/containernetworking/plugins/pkg/ns"
    "github.com/florianl/go-tc"
    "github.com/florianl/go-tc/core"
    "github.com/sirupsen/logrus"
        "github.com/mdlayher/netlink"
    "golang.org/x/sys/unix"
)

const (
    path_ns = "/var/run/netns"
)

// AddHTBToInterface  acts like
// $TC qdisc add dev {ifName} root handle 1:0 htb
func AddHTBToInterface(ifName string) error {
    ifByName, err := net.InterfaceByName(ifName)
    if err != nil {
        return err
    }

    qdiscHTB := createHTBObject(uint32(ifByName.Index), 
                            core.BuildHandle(0x1, 0x0), 
                            tc.HandleRoot, 
                            nil, &tc.HtbGlob{Version: 0x3, Rate2Quantum: 0xa,})

    tcnl, err := tc.Open(&tc.Config{})
    if err != nil {
        logrus.Errorf("err creating tcnl, err = %s", err)
        return err
    }
    defer tcnl.Close()
    if err := tcnl.SetOption(netlink.ExtendedAcknowledge, true); err != nil {
        logrus.Warnf("EXT_ACK set failed, err = %v", err)
    }

    if err := tcnl.Qdisc().Add(qdiscHTB); err != nil {
        return err
    }
    return nil
}

// AddHTBClass acts as
// $TC class add dev {ifName} parent {parent} classid {classid} htb rate {limit}
func AddHTBClass(ifName string, parent uint32, classid uint32, limit uint64) error {
    ifByName, err := net.InterfaceByName(ifName)
    if err != nil {
        logrus.Errorf("cannot find %s interface", ifName)
        return err
    }

    rate := limit   
    htbObject := 
        createHTBObject(uint32(ifByName.Index), 
                    classid,
                    parent,
                    &rate, &tc.HtbGlob{Version: 0x3, Rate2Quantum: 0xa,})

    tcnl, err := tc.Open(&tc.Config{})
    if err != nil {
        logrus.Errorf("err creating tcnl, err = %s", err)
        return err
    }
    defer tcnl.Close()
    if err := tcnl.SetOption(netlink.ExtendedAcknowledge, true); err != nil {
        logrus.Warnf("EXT_ACK set failed, err = %v", err)
    }
    if err := tcnl.Class().Add(htbObject); err != nil {
        return err
    }

    return nil
}

func createHTBObject(ifIndex uint32, handle uint32, parent uint32, rate *uint64, htbInit *tc.HtbGlob) *tc.Object {
    return &tc.Object{
        Msg: tc.Msg{
            Family:  unix.AF_UNSPEC,
            Ifindex: uint32(ifIndex),
            Handle:  handle,
            Parent:  parent,
            Info:    0,
        },
        Attribute: tc.Attribute{
            Kind: "htb",
            Htb: &tc.Htb{
                Init: htbInit,
                Rate64: rate,
            },
        },
    }
}

func createHTB() {

    ns1, err := ns.GetNS(path_ns + "/ns1")
    if err != nil {
        logrus.Fatalf("can not open ns1, err=%v", err)
    }

    err = ns1.Do(func(hostNS ns.NetNS) error {
        return AddHTBToInterface("v-ns1")
    })

    if err != nil {
        logrus.Errorf("ns1 tc get err, err = %s", err)
    }

}

func createClass() {
    ns1, err := ns.GetNS(path_ns + "/ns1")
    if err != nil {
        logrus.Fatalf("can not open ns1, err=%v", err)
    }

    err = ns1.Do(func(hostNS ns.NetNS) error {
        return AddHTBClass("v-ns1", core.BuildHandle(1,0), core.BuildHandle(1, 1), 100000)
    })

    if err != nil {
        logrus.Errorf("ns1 tc get err, err = %s", err)
    }
}

func main() {
    //createHTB()   // run successfully 
    createClass()

}

after running createHTB(), I use ip netns exec ns1 tc -s qdisc ls dev v-ns1 to check v-ns1, the result seems good:

qdisc htb 1: root refcnt 9 r2q 10 default 0 direct_packets_stat 5 direct_qlen 1000
 Sent 350 bytes 5 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0

but when running createClass(), I get the err err = netlink receive: invalid argument.

[florianl]

Thanks for your report. I will try to find some time in the coming days to take a look at it and come back.