appropriate use-case

[travisghansen]

Would this library be appropriate for the following operations:

• upsert a routing table
• replace a rule in the routing table created in step 1
• add (preferably in an idempotent fashion, no duplicates) ip rules to force specific traffic to use routing table in step 1

# create table
# test table exists
# exit 0   = exists
# exit 255 = not exists
ip route show table cilium
echo 20 cilium >> /etc/iproute2/rt_tables

# upsert routes to the table
# note that linux does a hash-based 3/5 tuple algorithm
# https://serverfault.com/questions/696675/multipath-routing-in-post-3-6-kernels
ip route replace default table cilium \
    nexthop via 172.28.4.130 weight 1 \
    nexthop via 172.28.4.131 weight 1

# add rule(s)
# test if the rule is already present? do not create duplicates
ip rule add from 172.28.42.0/24 lookup cilium
...

Thanks!

[mildsunrise]

yes, these actions (except creating the routing table) use rtnetlink, and rtnetlink is carred by netlink. so you can use this library to access the API. . what's more, the rtnetlink API in particular is specifically exposed in a high level way, complete with typescript bindings. check out RtNetlinkSocket. however, I see that the NEWRULE, GETRULE, etc. operations are not implemented.

[mildsunrise]

NEWRULE / GETRULE / DELRULE operations implemented in https://github.com/mildsunrise/node_netlink/commit/e978e06afbad6a639a2cf8a7badc58b918af6a4c, I'll release when I find time

[travisghansen]

Awesome! I’ll take a peek.

[mildsunrise]

in general you should be able to do anything ip can do, except maybe netns. the problem is that documentation on rtnetlink is super scarce. strace, kernel source code and iproute2 source code are your friends.

[travisghansen]

Attempting to use getRules results in this in my environment:

Error: Unexpected length (got 1, expected 2)
    at checkLength (/home/thansen/Projects/github/travisghansen/metallb-node-route-agent/node_modules/netlink/dist/structs.js:241:11)
    at Object.getU16 (/home/thansen/Projects/github/travisghansen/metallb-node-route-agent/node_modules/netlink/dist/structs.js:251:44)
    at Object.21 (/home/thansen/Projects/github/travisghansen/metallb-node-route-agent/node_modules/netlink/dist/rt/gen_structs.js:341:52)
    at /home/thansen/Projects/github/travisghansen/metallb-node-route-agent/node_modules/netlink/dist/structs.js:362:27
    at parseAttributes (/home/thansen/Projects/github/travisghansen/metallb-node-route-agent/node_modules/netlink/dist/structs.js:232:9)
    at Object.getObject (/home/thansen/Projects/github/travisghansen/metallb-node-route-agent/node_modules/netlink/dist/structs.js:359:5)
    at Object.parseRouteAttrs (/home/thansen/Projects/github/travisghansen/metallb-node-route-agent/node_modules/netlink/dist/rt/gen_structs.js:320:20)
    at Object.parseRouteMessage (/home/thansen/Projects/github/travisghansen/metallb-node-route-agent/node_modules/netlink/dist/rt/structs.js:115:22)
    at parseMessage (/home/thansen/Projects/github/travisghansen/metallb-node-route-agent/node_modules/netlink/dist/rt/structs.js:186:23)
    at /home/thansen/Projects/github/travisghansen/metallb-node-route-agent/node_modules/netlink/dist/rt/rt.js:75:56

[mildsunrise]

sorry, it seems the manpage had incorrect information about NEWRULE, GETRULE, DELRULE:

Add, delete, or retrieve a routing rule. Carries a struct rtmsg.

Which is either incorrect or outdated. According to kernel source they carry a fib_rule_hdr + attrs, defined here, not an rtmsg.

Therefore adding support for these operations requires more work, particularly transcribing the types in that header into types/rt.ts. I'll revert the commit. PRs are welcome, otherwise maybe I'll do it myself if I find some time.

[mildsunrise]

in the end I did it myself, since it was a short header. I've also released v0.2.1

[travisghansen]

Awesome! Both issues have cleared up for me now. This is all a bit lower-level than I tend to mess with so thanks for the patience answering some questions :)

How would I go about turning this: https://netlink.alba.sh/docs/interfaces/rt_gen_structs.RouteAttrs.html#multipath into their respective objects?

For example I've got this logged:

{
  kind: 'route',
  data: {
    family: 2,
    dstLen: 0,
    srcLen: 0,
    tos: 0,
    table: 20,
    protocol: 'BOOT',
    scope: 'UNIVERSE',
    type: 'UNICAST',
    flags: {}
  },
  attrs: {
    table: 20,
    multipath: <Buffer 10 00 00 63 02 00 00 00 08 00 05 00 ac 1d 00 01 10 00 00 63 02 00 00 00 08 00 05 00 ac 1d 00 03>
  }
}

which corresponds to this (what I'm currently using to get the feature):

ip -j -d route show table 20  | jq .
[
  {
    "type": "unicast",
    "dst": "default",
    "protocol": "boot",
    "scope": "global",
    "flags": [],
    "nexthops": [
      {
        "gateway": "172.29.0.1",
        "dev": "wlp0s20f3",
        "weight": 100,
        "flags": []
      },
      {
        "gateway": "172.29.0.3",
        "dev": "wlp0s20f3",
        "weight": 100,
        "flags": []
      }
    ]
  }
]

I'm unclear how I turn that multipath buffer into these: https://netlink.alba.sh/docs/modules/rt_structs.html#RouteNextHop

Also, out of curiosity, is it possible to receive a stream of events when new rules/routes are added/deleted/modified? If so is there an example floating around I can look at?

Thanks!

[mildsunrise]

To answer your first question: if I remember correctly, multipath isn't automatically parsed by the API because it's a "strange" type: a struct with a 'length' field (RouteNextHop), which IIUC is followed by RouteAttrs* if there's excess space. It's unusual to find stuff like that in Netlink, so the parsing system doesn't support it.

It's very poorly documented, but the code to parse it would be something like:

function parseMultipath(input) {
  const result = []
  while (input.length) {
    // Parse struct at the start
    const data = parseRouteNextHop( input.slice(0, __LENGTH_RouteNextHop) )
    // Parse excess data as attrs
    if (input.length < data.len)
      throw Error(`invalid rtnexthop length ${data.len}`)
    const attrs = parseRouteAttrs(input.slice(__LENGTH_RouteNextHop, data.len))
    // Remove consumed data from input (plus alignment)
    input = input.slice(data.len + (-data.len & 3))
    // Add to result
    delete data.len // not useful to caller
    result.push({ data, attrs })
  }
  return result
}

By the way, as you can see, weight is reported as data.hops + 1 according to iproute2's source code.

(*): in fact only gateway is currently allowed there AFAIK.

[travisghansen]

Got it. Thanks for the help!

[mildsunrise]

To answer the second question:

Also, out of curiosity, is it possible to receive a stream of events when new rules/routes are added/deleted/modified? If so is there an example floating around I can look at?

Yes, this was one of Netlink's objectives. In Netlink, both parties send can messages of a certain type. Sending NEWROUTE indicates a request to create a route, and receiving NEWROUTE from the kernel is a notification that a new route was added.

Messages may be (1) unsolicited, or (2) received in response to a previous request. For the latter, the message is returned from the promise as you know. Unsolicited messages are emitted as message events. But in order to receive unsolicited messages from a Netlink API, you usually have to subscribe to a multicast group. Rtnetlink has many multicast groups.

You can find an example here (beware this is an old version of the README, the latest version is a bit different since I'm retouching the API for the next version). Make sure you're using at least v0.2.2 since I fixed a little bug regarding unsolicited messages.

[travisghansen]

Awesome! I’ll give it a try. Thanks again for the answers!

[travisghansen]

Yup, got the multicast groups going just fine locally with v0.2.2!