TCP checksums not making it all the way through to the TCP stack?

[erikarn]

Hi!

I have an odd problem in local testing - it seems hardware TCP checksums aren't making it up to the TCP stack in libuinet and it's doing software checksumming. Is this known?

[root@foo] ~# dtrace -n 'profile-1997 /arg1/ { @[execname, ustack()] = count(); } END { trunc(@, 3); }' dtrace: description 'profile-1997 ' matched 2 probes ^C CPU ID FUNCTION:NAME 15 2 :END

blockd
libc.so.7memcpy+0x20 blockdnetisr_dispatch_src+0x6c blockdif_netmap_receive+0x332 blockdpthread_start_routine+0xdb libthr.so.30x8022954a5 166 blockd blockdether_nh_input+0x85 blockdnetisr_dispatch_src+0x6c blockdif_netmap_receive+0x332 blockdpthread_start_routine+0xdb libthr.so.30x8022954a5 349 blockd
blockdin_cksumdata+0xdf blockdin_cksum_skip+0x69 blockdtcp_input+0xc1 blockdip_input+0x8e blockdnetisr_dispatch_src+0x6c blockdether_nh_input+0x61c blockdnetisr_dispatch_src+0x6c blockdif_netmap_receive+0x332 blockdpthread_start_routine+0xdb libthr.so.30x8022954a5 835

[caladri]

I don't think netmap has any way to indicate that a packet has a hardware checksum yet; presumably the flags field in struct netmap_slot would need to track this, as well as possibly tracking it on the transmit side to indicate that the hardware needs to checksum on tx. In other similar systems, you get the actual bits from the hardware to report this, but netmap's abstraction means that the relevant flags need to propagate by hand.

(And you'll definitely want to ensure they follow the packet from rx to tx in your application as well, especially if you're using VALE to connect multiple applications together.)

This is a netmap issue and not a libuinet one, unless you want a knob to tell libuinet to assume all packets received from netmap have had hardware checksum done. Which is not unreasonable for some hardware and configurations, but I'm glad it's not the default. I use netmap with hardware that deliberately doesn't do any packet checking routinely.

[erikarn]

Yeah, I dug into it a bit more and I agree.

I think it'd be useful to get TCP checksums propagated down/up the stack but yes, we'd have to start by extending netmap to include all that metadata.

[caladri]

Do you think you'd find value in a mode whereby libuinet could be configured to flag all packets received from netmap as having valid checksums? Or should this be closed until netmap grows support?

[erikarn]

I think we should keep this open for the time being. I would like to see netmap grow hardware transmit/receive checksum support so libuinet doesn't have to do it. Otherwise a whole lot of CPU time is spent calculating checksums.

[gonzopancho]

There is (unreleased) code in netmap-fwd to do this. I'll get a copy to Patrick.