Wrong byte order for loopback pcaps

[josephcsible]

Brief description

The pcap file format supports two different link-layer header types for loopback: NULL (0), in which the header is in host byte order, and LOOP (108), in which the header is in network byte order.

When scapy reads loopback pcaps, it reads the header in host byte order whether the header type is 0 or 108. When scapy writes loopback pcaps, it stores 108 as the header type but stores the header in host byte order anyway.

This results in Wireshark and other tools saying "Family: Unknown (33554432)" and not decoding the IP packet inside. The code to read pcaps is broken in the same way, so scapy can read its own malformed pcaps but not correctly-formatted ones of type 108.

Scapy version

2.4.5rc1.dev259

Python version

3.6.8

Operating system

RHEL 7.9

Additional environment information

No response

How to reproduce

Do wrpcap('loopbackbug.pcap', [Loopback()/IP()/ICMP()]) in scapy and then tshark -Vr loopbackbug.pcap from the shell.

Actual result

It will print "Family: Unknown (33554432)" and not be able to decode the IP or ICMP headers.

Expected result

It should print "Family: IP (2)" and then decode the IP and ICMP headers.

Related resources

https://www.tcpdump.org/linktypes.html documents each link header type.

[guedou]

Thanks for reporting the issue. I was able to reproduce it.

[stryngs]

@josephcsible Do you have an example pcap where a Loopback type field exists in Wireshark? As an example I did a ping to 127.0.0.1 and a sniff of lo.

I think Wireshark is doing what it was designed to do in this case. here is why:

In [12]: f=Loopback()/IP()/ICMP()

In [13]: z = Loopback(f.build())

In [14]: z==f
Out[14]: False

In [15]: f
Out[15]: <Loopback  type=IPv4 |<IP  frag=0 proto=icmp |<ICMP  |>>>

In [16]: z
Out[16]: <Loopback  type=IPv4 |<IP  version=4 ihl=5 tos=0x0 len=28 id=1 flags= frag=0 ttl=64 proto=icmp chksum=0x7cde src=127.0.0.1 dst=127.0.0.1 |<ICMP  type=echo-request code=0 chksum=0xf7ff id=0x0 seq=0x0 |>>>

In [17]: f.show()
###[ Loopback ]### 
  type      = IPv4
###[ IP ]### 
     version   = 4
     ihl       = None
     tos       = 0x0
     len       = None
     id        = 1
     flags     = 
     frag      = 0
     ttl       = 64
     proto     = icmp
     chksum    = None
     src       = 127.0.0.1
     dst       = 127.0.0.1
     \options   \
###[ ICMP ]### 
        type      = echo-request
        code      = 0
        chksum    = None
        id        = 0x0
        seq       = 0x0
        unused    = ''

You expect the above, but Wireshark has no idea what to do with it. After all, lo understands TCP... Thus when we sniff:

In [7]: pkts[0].layers()
Out[7]: 
[scapy.layers.l2.Ether,
 scapy.layers.inet.IP,
 scapy.layers.inet.ICMP,
 scapy.packet.Raw]

Without the upper layers of the onion, Wireshark is confused, as it should be. It needs hints and I do not believe it has this ability.

[josephcsible]

Do you have an example pcap where a Loopback type field exists in Wireshark?

Do a capture on lo with Wireshark. Verify the header type is either NULL or LOOP. Verify that Wireshark can decode the resulting .pcap correctly. Go into scapy and rdpcap that file, then immediately wrpcap the result into a new file. Try to open the new file with Wireshark.

[stryngs]

Wireshark sniff results in the outer layer of Ether, as I mentioned above. Good test idea. Tcpdump also confirms.

In [1]: from scapy.all import *
pkts
In [2]: pkts = rdpcap('wireshark_icmp_lo.pcap')

In [3]: pkts[0]
Out[3]: <Ether  dst=00:00:00:00:00:00 src=00:00:00:00:00:00 type=IPv4 |<IP  version=4 ihl=5 tos=0x0 len=84 id=11719 flags=DF frag=0 ttl=64 proto=icmp chksum=0xee0 src=127.0.0.1 dst=127.0.0.1 |<ICMP  type=echo-request code=0 chksum=0x9675 id=0x2 seq=0x1 unused='' |<Raw  load='i\\xfd\\x8ac\x00\x00\x00\x00\\xaeS\x00\x00\x00\x00\x00\x00\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f !"#$%&\'()*+,-./01234567' |>>>>

Without an Ether() type layer from Scapy and perhaps IP, etc... Wireshark will not understand what you're trying to do, not without a change to Wireshark source code.

I would say this is Expected Scapy Behavior @guedou

[stryngs]

What are you trying to achieve from a code perspective @josephcsible ?

[josephcsible]

How could this be considered expected behavior? It's unambiguously violating the PCAP spec.

[stryngs]

You're giving Wireshark one layer to decipher. Wireshark doesn't to my knowledge know how to do such a thing. Had you properly encapsulated your frame or packet, then Wireshark would act properly. I wasn't even aware of the Loopback layer until this issue. So, I tested. As expected, Ether() is needed.

If you can cite a known pcap that shows a Loopback layer in Wireshark then you may be able to achieve what you want. Without an example to go by, you're just blaming Scapy for what is Expected Wireshark Behavior.

[josephcsible]

loopbackbug2.zip

[stryngs]

When I'm wrong I'm wrong.

Thank you for providing the pcap @josephcsible. This helps people to debug where otherwise guesses have to be made. I tried to come up with a workaround for you but could not.

Good find!

[stryngs]

I've never modified scapy with respect to how it dissects but I would guess this would be a good place in the source: https://github.com/secdev/scapy/blob/master/scapy/layers/l2.py#L661 - 676.

Grep may be a pal here too:

/site-packages/scapy$ grep -RHn Loopback 2>/dev/null
layers/lltd.py:402:            24: "softwareLoopback",
layers/inet6.py:50:from scapy.layers.l2 import CookedLinux, Ether, GRE, Loopback, SNAP
layers/inet6.py:4074:bind_layers(Loopback, IPv6, type=socket.AF_INET6)

layers/inet.py:27:    Loopback
layers/inet.py:1027:bind_bottom_up(Loopback, IP, type=0)
layers/inet.py:1028:bind_layers(Loopback, IP, type=socket.AF_INET)
layers/l2.py:618:class Loopback(Packet):
layers/l2.py:621:    name = "Loopback"
layers/l2.py:672:conf.l2types.register(DLT_LOOP, Loopback)
layers/l2.py:673:conf.l2types.register_num2layer(DLT_NULL, Loopback)
contrib/automotive/bmw/definitions.py:4853:UDS_RC.routineControlIdentifiers[0x0303] = "DiagLoopbackStart"
contrib/homeplugav.py:77:                0xA048: "'Loopback Request'",
contrib/homeplugav.py:78:                0xA049: "'Loopback Request Confirmation'",
contrib/homeplugav.py:334:class LoopbackRequest(Packet):
contrib/homeplugav.py:335:    name = "LoopbackRequest"
contrib/homeplugav.py:342:class LoopbackConfirmation(Packet):
contrib/homeplugav.py:343:    name = "LoopbackConfirmation"
contrib/homeplugav.py:1435:bind_layers(HomePlugAV, LoopbackRequest, HPtype=0xA048)
contrib/homeplugav.py:1436:bind_layers(HomePlugAV, LoopbackConfirmation, HPtype=0xA049)
arch/libpcap.py:222:                conf.loopback_name = conf.loopback_name = "Npcap Loopback Adapter"  # noqa: E501
arch/libpcap.py:451:            # Makes send detects when it should add Loopback(), Dot11... instead of Ether()  # noqa: E501
arch/bpf/supersocket.py:27:from scapy.layers.l2 import Loopback
arch/bpf/supersocket.py:393:        if self.guessed_cls == Loopback:
arch/windows/structures.py:545:                ("Loopback", BOOLEAN),
arch/windows/__init__.py:54:NPCAP_LOOPBACK_NAME = r"\Device\NPF_Loopback"
arch/windows/__init__.py:60:            conf.loopback_name = "Microsoft KM-TEST Loopback Adapter"
arch/windows/__init__.py:62:            conf.loopback_name = "Microsoft Loopback Adapter"
arch/windows/__init__.py:64:        conf.loopback_name = "Microsoft Loopback Adapter"
arch/windows/__init__.py:105:    LoopbackAdapter, LoopbackSupport, NdisImPlatformBindingOptions, VlanSupport
arch/windows/__init__.py:568:            # Detect Loopback interface
arch/windows/__init__.py:569:            if "Loopback" in i['name']:

[guedou]

I will continue the investigations. This fixes the issue wrpcap('loopbackbug.pcap', [Loopback()/IP()/ICMP()], linktype=DLT_NULL)