payload is not the same as before when the packet is written to a new pcap file

[zhou121]

for ts,buf in pcap: eth=dpkt.ethernet.Ethernet(buf) print(eth.data.len) writer.writepkt(eth,now_time) print(eth.data.len)

the first output is 163 the second output is 124

I find the packet's payload in the new pcap changes. But I do not modify the payload. Why the packet can not have the same payload as in the old pcap?

[obormot]

please attach the pcap (preferably just the 1 packet that causes the issue)

[zhou121]

p2pbox3.2011040311.pcap.clean.zip

infile = './p2pbox3.2011040311.pcap.clean.pcap'
outfile = './test.pcap'
fr = open(infile,'rb')
pcap = dpkt.pcap.Reader(fr)
fw = open(outfile,'wb')
writer = dpkt.pcap.Writer(fw)

for ts,buf in pcap:
    eth=dpkt.ethernet.Ethernet(buf)
    if eth.type!=0x0800 or (eth.data.p!=17 and eth.data.p!=6):
        continue
    print(eth.data.len)
    writer.writepkt(eth,1234567890.000000)
    print(eth.data.len)
    break
fr.close()
fw.close()

[obormot]

This appears to be a long standing issue with dpkt, where ip.len changes when it is serialized (see https://github.com/kbandla/dpkt/issues/279)

This is from the pcap attached here:

In [3]: print(dpkt.hexdump(s))
  0000:  00 1a a0 78 8b 3d 00 1a a0 77 1f 9d 08 00 45 00  ...x.=...w....E.
  0016:  00 a3 b6 7a 00 00 80 11 5e 6f c0 a8 03 02 97 47  ...z....^o.....G
  0032:  ca 6e c7 38 64 df 00 8f 4a fa 26 d8 15 be d9 42  .n.8d...J.&....B
  0048:  ae 66 e1 ce 14 5f 06 79 4b 13 02 ad a4 8b 69 1c  .f..._.yK.....i.
  0064:  7a f6 d5 3d 45 aa ba cd 24 77 c2 e7 5f 6a cc b5  z..=E...$w.._j..
  0080:  1f 21 fa 62 f0 f3 32 e1 e4 f0 20 1f 47 61 ec bc  .!.b..2... .Ga..
  0096:  b1 0e 6c f0 b8 6d 7f 96 9b 35 03 a1 79 05 c5 fd  ..l..m...5..y...
  0112:  2a f7 fa 35 e3 0e 04 d0 c7 4e 94 72 3d 07 5a a8  *..5.....N.r=.Z.
  0128:  53 2a 5d 03 f7 04 c4 a8 b8 a1                    S*].......

In [4]: dpkt.ethernet.Ethernet(s).ip.len
Out[4]: 163

In [5]: dpkt.ethernet.Ethernet(bytes(dpkt.ethernet.Ethernet(s))).ip.len
Out[5]: 124

^^ this should be the same, but it changes when serialized by calling bytes(); same happens on pcap write

[obormot]

this should now be fixed in the master branch

[zhou121]

this should now be fixed in the master branch

It is true that the length of the output is currently the same both times. But I found that the generated test.pcap opens with wireshark with the following error：

IP : [Expert Info (Error/Protocol): IPv4 total length exceeds packet length (124 bytes)] udp: [Expert Info (Error/Malformed): Bad length value 143 > IP payload length]

It seems that the value shown for the total length field is not the actual packet size.

[obormot]

Yeah the packets in the original pcap appear truncated, so that should be expected. After the change dpkt will keep the original length, instead of recalculating it. If you'd like dpkt to recalculate the length, then set ip.len to zero before writing the pcap file. E.g.

eth = dpkt.ethernet.Ethernet(buf)
if isinstance(eth.data, dpkt.ip.IP):
    eth.ip.len = 0
...
writer.writepkt(eth)

this will go back to the behavior seen before the change; ip.len will be recalculated and Wireshark will be happy.

[zhou121]

I don't think the packet was truncated in the original pcap, but rather it seems that the packet was truncated in the newly generated test.pcap. So, after recalculating the package length, the package length does not match the length of that package in the original pcap again.

[obormot]

Not a single packet in the original packet is longer than 138 bytes, which indicates they were truncated

[obormot]

Note when you open the original pcap with Wireshark (1st packet), it says: Frame 1: 177 bytes on wire (1416 bits), 138 bytes captured (1104 bits) indicating the capture was truncated.

When you open the output pcap, the packet contents is exactly the same, but the description says Frame 1: 138 bytes on wire (1104 bits), 138 bytes captured (1104 bits) and this time Wireshark throws warnings saying the IP length and UDP length are bogus.

The difference is in the field called "caplen" (capture length) in the pcap header. Since dpkt doesn't preserve the original caplen from the input pcap, it calculates the new caplen when saving the output pcap, based on the size of the data it has, which is 138 bytes. As a result Wireshark no longer sees the typical truncation that it is able to recognize in the original pcap, but an "invalid" truncation, therefore it throws warnings.