Making use of rte_pktmbuf_headroom to enlarge/shrink a WritablePacket

[kthkaya]

Hi,

I am working on a stateful/bidirectional 64 translator with DPDK support and I am wondering whether one can come up with a more efficient way than the following using the headroom. https://github.com/kohler/click/blob/a46041412238420b26ca11ec88c7981942fb8b45/elements/ip6/protocoltranslator46.cc#L48-L58

and

https://github.com/kohler/click/blob/a46041412238420b26ca11ec88c7981942fb8b45/elements/ip6/protocoltranslator64.cc#L87-L88

Assuming translation is 4 to 6 for Packet *p,

click_ip6 ip6h = populate fields from mappings and/or Packet p; WritablePacket *wp = p->uniqueify(); (I assume I can't act on p directly?)

1. Make 20 bytes available in the headroom, return wp->data() pointing to the beginning of the 20 bytes.
2. memmove(wp->data, wp->data+20, ETHERHEADERLEN).
3. Assuming that ETHERHEADERLEN is 14, wp looks like following at this state
   • Eth header is at wp->data[0] to wp->data[13]
   • wp->data[14] to wp->data[34] is empty and wp->data[34] to wp->data[54] is the ipv4 header that we dont care about anymore.
4. memcopy(wp->data[34], ip6h, IPV6HEADERLEN)
5. Update l4 header offset in wp.

For 6 to 4 it is pretty much the opposite, returning 20 bytes to the headroom. Do you think this would be more resource efficient, or just a tidier code?

And then there is this comment https://github.com/kohler/click/blob/a46041412238420b26ca11ec88c7981942fb8b45/include/click/packet.hh#L969-L973

How to maintain that requirement headroom > length when performing the operations above?

Thanks

[tbarbette]

This element does not really depend on DPDK, right?

Though, DPDK packets comes with a default headroom of 128bytes which is very convenient for encapsulation purposes. You may change this amount of bytes in dpdkdevice.hh if needed.

You should not need a memove, ToDPDKDevice (and other) will take p->data() as the beginning of the packet.

If you build carefully your pipeline to have unshared packets arriving in your element, uniqueify should in fact only cast the packet, and your headroom will be already large enough. So you can simply call p->push() to move the data pointer from the amount bytes needed.

If you want to build something resilient, the code would look like this, where needed is the amount of headroom needed :

WritablePacket* wp;
if (p->shared()) {
  int missing = needed - p->headroom();
  if (missing < 0)
    missing = 0;
  wp = p->expensive_uniqueify(missing, 0, true);
} else {
  wp = static_cast<WritablePacket*>(p);
}
wp = wp->push(needed);

In this way there will be only one packet copy or move, and only if needed. p->data() will point towards the amount of byte needed before the packets. It's up to you to build a new ethernet frame, then the IP headers etc.

[kthkaya]

Thanks for the fast response Tom!

This element does not really depend on DPDK, right?

At this point, I am developing the element for DPDK dataplane only, if this is what you meant by depend.

You should not need a memove, ToDPDKDevice (and other) will take p->data() as the beginning of the packet.

But it will point to needed bytes of free space after wp->push(needed) and the actual packet is now at wp->data()[needed] right? Therefore I memmove the ethernet header to the beginning of wp->data() so that I have the needed free space after the ethernet header and before the network header (i.e. between wp->data()[14] and wp->data()[34] ). Is this correct?

[tbarbette]

Yes it's correct, but it's often a mistake to keep the ethernet header as it (though this is often seen...). It is correct only if you want a "plug in the cable" totally transparent solution, which will require to set the device in promisc mode. In most cases you'll want to "Unstrip(14)" to decapsulate the ethernet header, encap the IP things in your 6-to-4 element, keeping everything at the IP level and then use EtherEncap to encapsulate the 6-to-4 packet with a new correct header, that is with the right source and destination mac address. This also makes your element more modular as it would work also with IP-over-ATM for example.

[kthkaya]

Thanks for the heads up! Keeping the ethernet header is intentional because IPv6 elements that can handle L2 resolution down the pipeline (IP6 neighbor discovery elements) are not DPDK enabled/optimized.