IPv4 Checksum fails (due to tcp offloading)

[josemic]

I have the case that wireshark and pkt calculate (the identical) checksum error, but my linux pc acknowledges the packet. Thus either wireshark and pkt have both the same error, pcap has an error or my Linux-PC does not calculate the TCP checksum before acknowledging the packet.

Fame = <<0,37,34,169,124,93,156,199,166,109,119,220,8,0,69,0,5,220,122,52,64,
          0,247,6,63,111,193,99,144,85,192,168,178,22,0,80,198,169,201,148,4,
          83,16,103,129,8,128,24,17,80,202,70,0,0,1,1,8,10,229,143,96,87,0,21,
          41,232,72,84,84,80,47,49,46,49,32,50,48,48,32,79,75,13,10,68,97,116,
          101,58,32,87,101,100,44,32,50,51,32,79,99,116,32,50,48,49,51,32,49,
          51,58,52,51,58,53,48,32,71,77,84,13,10,83,101,114,118,101,114,58,32,
          65,112,97,99,104,101,13,10,69,120,112,105,114,101,115,58,32,87,101,
          100,44,32,50,51,32,79,99,116,32,50,48,49,51,32,49,51,58,52,56,58,53,
          49,32,71,77,84,13,10,86,97,114,121,58,32,85,115,101,114,45,65,103,
          101,110,116,44,65,99,99,101,112,116,45,69,110,99,111,100,105,110,103,
          13,10,67,97,99,104,101,45,99,111,110,116,114,111,108,58,32,109,97,
          120,45,97,103,101,61,51,48,48,13,10,88,45,67,111,98,98,108,101,114,
          58,32,111,99,116,111,49,52,46,104,101,105,115,101,46,100,101,13,10,
          67,111,110,110,101,99,116,105,111,110,58,32,99,108,111,115,101,13,10,
          84,114,97,110,115,102,101,114,45,69,110,99,111,100,105,110,103,58,32,
          99,104,117,110,107,101,100,13,10,67,111,110,116,101,110,116,45,84,
          121,112,101,58,32,116,101,120,116,47,104,116,109,108,59,32,99,104,97,
          114,115,101,116,61,117,116,102,45,56,13,10,13,10,49,97,48,98,54,13,
          10,60,33,68,79,67,84,89,80,69,32,104,116,109,108,62,10,60,104,116,
          109,108,32,108,97,110,103,61,34,100,101,34,62,10,10,60,104,101,97,
          100,62,10,10,32,32,32,32,60,116,105,116,108,101,62,73,84,45,78,101,
          119,115,44,32,99,39,116,44,32,105,88,44,32,84,101,99,104,110,111,108,
          111,103,121,32,82,101,118,105,101,119,44,32,84,101,108,101,112,111,
          108,105,115,32,124,32,104,101,105,115,101,32,111,110,108,105,110,101,
          60,47,116,105,116,108,101,62,10,32,32,32,32,32,32,32,32,60,109,101,
          116,97,32,110,97,109,101,61,34,100,101,115,99,114,105,112,116,105,
          111,110,34,32,99,111,110,116,101,110,116,61,34,78,101,119,115,32,117,
          110,100,32,70,111,114,101,110,32,122,117,32,67,111,109,112,117,116,
          101,114,44,32,73,84,44,32,87,105,115,115,101,110,115,99,104,97,102,
          116,44,32,77,101,100,105,101,110,32,117,110,100,32,80,111,108,105,
          116,105,107,46,32,80,114,101,105,115,118,101,114,103,108,101,105,99,
          104,32,118,111,110,32,72,97,114,100,119,97,114,101,32,117,110,100,32,
          83,111,102,116,119,97,114,101,32,115,111,119,105,101,32,68,111,119,
          110,108,111,97,100,115,32,98,101,105,109,32,72,101,105,115,101,32,90,
          101,105,116,115,99,104,114,105,102,116,101,110,32,86,101,114,108,97,
          103,46,34,32,47,62,10,32,32,32,32,32,32,32,32,32,32,32,32,60,109,101,
          116,97,32,110,97,109,101,61,34,107,101,121,119,111,114,100,115,34,32,
          99,111,110,116,101,110,116,61,34,104,101,105,115,101,32,111,110,108,
          105,110,101,44,32,99,39,116,44,32,105,88,44,32,84,101,99,104,110,111,
          108,111,103,121,32,82,101,118,105,101,119,44,32,78,101,119,115,116,
          105,99,107,101,114,44,32,84,101,108,101,112,111,108,105,115,44,32,83,
          101,99,117,114,105,116,121,44,32,78,101,116,122,101,34,32,47,62,10,
          10,32,32,32,32,10,10,10,10,60,109,101,116,97,32,99,104,97,114,115,
          101,116,61,34,117,116,102,45,56,34,62,10,60,109,101,116,97,32,110,97,
          109,101,61,34,112,117,98,108,105,115,104,101,114,34,32,99,111,110,
          116,101,110,116,61,34,72,101,105,115,101,32,90,101,105,116,115,99,
          104,114,105,102,116,101,110,32,86,101,114,108,97,103,34,32,47,62,10,
          60,109,101,116,97,32,110,97,109,101,61,34,118,105,101,119,112,111,
          114,116,34,32,99,111,110,116,101,110,116,61,34,119,105,100,116,104,
          61,100,101,118,105,99,101,45,119,105,100,116,104,44,32,105,110,105,
          116,105,97,108,45,115,99,97,108,101,61,49,46,48,34,32,47,62,60,108,
          105,110,107,32,114,101,108,61,34,104,111,109,101,34,32,116,121,112,
          101,61,34,116,101,120,116,47,104,116,109,108,34,32,116,105,116,108,
          101,61,34,83,116,97,114,116,115,101,105,116,101,34,32,104,114,101,
          102,61,34,47,34,32,47,62,10,60,108,105,110,107,32,114,101,108,61,34,
          99,111,112,121,114,105,103,104,116,34,32,116,105,116,108,101,61,34,
          67,111,112,121,114,105,103,104,116,34,32,104,114,101,102,61,34,47,73,
          109,112,114,101,115,115,117,109,45,52,56,54,50,46,104,116,109,108,34,
          32,47,62,32,32,32,32,32,10,32,32,32,32,10,32,32,32,32,32,32,32,32,60,
          33,45,45,103,111,111,103,108,101,111,102,102,58,32,97,108,108,45,45,
          62,10,32,32,32,32,32,32,32,32,60,109,101,116,97,32,110,97,109,101,61,
          34,116,119,105,116,116,101,114,58,99,97,114,100,34,32,32,32,99,111,
          110,116,101,110,116,61,34,115,117,109,109,97,114,121,34,32,47,62,10,
          32,32,32,32,32,32,32,32,60,109,101,116,97,32,110,97,109,101,61,34,
          116,119,105,116,116,101,114,58,115,105,116,101,34,32,32,32,99,111,
          110,116,101,110,116,61,34,64,104,101,105,115,101,111,110,108,105,110,
          101,34,32,47,62,10,32,32,32,32,32,32,32,32,60,109,101,116,97,32,110,
          97,109,101,61,34,116,119,105,116,116,101,114,58,100,111,109,97,105,
          110,34,32,99,111,110,116,101,110,116,61,34,104,101,105,115,101,46,
          100,101,34,32,47,62,10,32,32,32,32,60,109,101,116,97,32,112,114,111,
          112,101,114,116,121,61,34,102,98,58,112,97,103,101,95,105,100,34,32,
          32,32,32,32,99,111,110,116,101,110,116,61,34,51,51,51,57,57,50,51,54,
          55,51,49,55,34,32,47,62,10,32,32,32,32,60,109,101,116,97,32,112,114,
          111,112,101,114,116,121,61,34,111,103,58,116,105,116,108,101,34,32,
          32,32,32,32,32,32,99,111,110,116,101,110,116,61,34,73,84,45,78,101,
          119,115,44,32,99,38,35,51,57,59,116,44,32,105,88,44,32,84,101,99,104,
          110,111,108,111,103,121,32,82,101,118,105,101,119,44,32,84,101,108,
          101,112,111,108,105,115,34,32,47,62,10,32,32,32,32,60,109,101,116,97,
          32,112,114,111,112,101,114,116>>

[#ether{}, #ipv4{sum = IPSum} = IP, #tcp{sum = TCPSum} = TCP, Payload] = pkt:decapsulate(Frame).

[#ether{dhost = <<0,37,34,169,124,93>>,
        shost = <<156,199,166,109,119,220>>,
        type = 2048,crc = 0},
 #ipv4{v = 4,hl = 5,tos = 0,len = 1500,id = 31284,df = 1,
       mf = 0,off = 0,ttl = 247,p = 6,sum = 16239,
       saddr = {193,99,144,85},
       daddr = {192,168,178,22},
       opt = <<>>},
 #tcp{sport = 80,dport = 50857,seqno = 3381920851,
      ackno = 275218696,off = 8,cwr = 0,ece = 0,urg = 0,ack = 1,
      psh = 1,rst = 0,syn = 0,fin = 0,win = 4432,sum = 51782,
      urp = 0,
      opt = <<1,1,8,10,229,143,96,87,...>>},
 <<"HTTP/1.1 200 OK\r\nDate: Wed, 23 Oct 2013 13:43:50 GMT\r\nServer: Apache\r\nExpires: Wed, 23 Oct 2013 13:4"...>>]

> pkt:makesum([IP, TCP#tcp{sum = 0}, Payload]).
26846
> IPSum = pkt:makesum(IP#ipv4{sum = 0}).                    
16239

The pcap file is available upon request. Any ideas?

[msantos]

Might be due to tcp checksum offloading. You can try disabling it:

ethtool --show-offload <dev>
ethtool --offload rx off
ethtool --offload tx off
<...>

Is the packet in your wireshark screenshot the same one in your example? The checksums seem to be different.

On another topic: I added a new version of decapsulate/1 to pkt: pkt/decode/1,2. I will push it in a bit.

decapsulate/1 will now crash on any error. Use it when you only care about valid packets. You can catch the crash:

Packet = try pkt:decapsulate(Frame)
catch
    error:_ -> invalid_packet
end

decode/1,2 looks like:

case pkt:decode(Frame) of % can also specify the protocol pkt:decode(tcp, Frame)
    {ok, [#ether{}, #ipv4{}, #tcp{}, Payload]} ->
        Payload;
    {error, [#ether{}, #ipv4{}], {tcp, Data} = Failed} ->
       ... 

This should deal with truncated packets using the snaplen option or malformed packets. Sound ok?

[josemic]

• The packages should be the same, but I have activated relative sequence numbers in Wireshark. That is why the sequence numbers do not match.: Here the tcp checksum in hex and dec: 1> 16#68de. 26846
• About tcp checksum offloading: You are right. Here is a nice blog: http://securityonion.blogspot.de/2011/10/when-is-full-packet-capture-not-full.html To be able to track the sequence numbers and ACKs correctly from both sides of the connection on e.g. Ubuntu 13.10 I had to do the following:

sudo ethtool -K eth0 tx-checksum-ip4 off
sudo ethtool -K eth0 tso off

• decode: Yes. It sounds fine.

[msantos]

Thanks for the link!