Fields after payload

[william-r-dieter]

Does Scapy have a way to have fields after the payload? For example, Ethernet and 802.11 have an FCS field with a CRC of the whole frame.

I am capturing in monitor mode with a NIC that gives the FCS of received packets in the 802.11 headers and I want to be able to check the FCS. Wireshark tells me that about 10% of the packets have bad FCS, and I would like to handle frames with bad FCS differently.

I looked in scapy/layers/dot11.py and scapy/layers/l2.py and could not find an example. If there is some other layer with a field after the payload, maybe I could do the same thing in Dot11...

[p-l-]

Hi,

I don't really think so, at least for now. Could you share a capture file so that I can have a look? Attach it publicly if that's an option for you, if not you can drop me an e-mail.

[william-r-dieter]

Attached is a short pcap with just five packets. You can find the FCS in wireshark in the 802.11 packet.

I can work around not having the FCS in the 802.11 layer, by calculating it manually, for example:

cap = rdpcap('short-seq.pcapng.gz')
# Extract the FCS for each frame
fcs = [struct.unpack("<L", bytes(p[Dot11].payload)[-4:])[0] for p in cap]
# Compute the FCS over each frame
crc = [crc32(bytes(p[Dot11])[:-4]) & 0xFFFFFFFFL for p in cap]

In this capture, all the FCSs match the CRCs, but that is not always true. Computing it this way is a little inefficient, because each packet gets dissected in rdpcap (or PcapReader for bigger files) then re-serialized to compute extract the FCS and compute the CRC. I then modify the packet using Scapy's manipulators, serialize the packet to compute the CRC and write it into the payload. Then I use PcapWriter to write the packet to an output PCAP file, which causes it to be serialized again.

If there was a way to represent a field at the end of the payload, I could modify the Dot11 dissector to not only store the FCS, but also compute whether it is valid, saving a few serialization round trips. I would also like to have a way to say whether the FCS should be computed or taken the payload when it is written. That way, if the FCS was invalid on input, I can make sure it stays invalid on output.

short-seq.pcapng.gz

[stryngs]

I agree with @william-r-dieter that something should be done about the way FCS is handled for frames. There is a way I've been manually doing it, but it would be nice if it was a built-in per se.

[william-r-dieter]

Right now, fields always get added to the end of the header. An API that would be nice would be to add an optional location arg to the Field constructor (and all its children). The argument would take a location parameter to say whether the field should be added to the head or the tail of the packet. It would default to head when not present, so existing code would not change.

For example:

fields_desc = [
    BitField("subtype", 0, 4),
    BitEnumField("type", 0, 2, ["Management", "Control", "Data",
                                "Reserved"]),
    BitField("proto", 0, 2),
    FlagsField("FCfield", 0, 8, ["to-DS", "from-DS", "MF", "retry",
                                 "pw-mgt", "MD", "wep", "order"]),
    ShortField("ID",0),
    MACField("addr1", ETHER_ANY),
    ConditionalField(
        MACField("addr2", ETHER_ANY),
        lambda pkt: (pkt.type != 1 or
                     pkt.subtype in [0x9, 0xb, 0xa, 0xe, 0xf]),
    ),
    ConditionalField(
        MACField("addr3", ETHER_ANY),
        lambda pkt: pkt.type in [0, 2],
    ),
    ConditionalField(LEShortField("SC", 0), lambda pkt: pkt.type != 1),
    ConditionalField(
        MACField("addr4", ETHER_ANY),
        lambda pkt: (pkt.type == 2 and
                     pkt.FCfield & 3 == 3),  ## from-DS+to-DS
    ),
    # New field for FCS at end of frame
    LEIntField("fcs", 0, loc='tail'),
]

The first tail field is added to the end of the layer. subsequent tail fields would be added preceding the existing tail fields (that is, they grow to the front of the packet) with the payload between the head and tail fields.

Not sure how much code would be impacted to do the dissection and serialization.

[p-l-]

OK thanks for the clarification.

[ChrisMGeiger]

I implemented similar protocol that put the checksum at the end of the packet. I achieved this with some inspiration from the Bluetooth LE protocol CRC field. In short, the calls to pre_dissect() and post_build() rearrange the packet to place the tail (CRC) field at the end of the packet instead of in the header.

[gpotter2]

FTR, support for Dot11 FCS was added to scapy. I however agree that such a field would be handy. Feel free to ping if you have an example case we could look into (not the already implemented layers)

[guedou]

Issue closed as FCSField is now in Scapy.

https://scapy.readthedocs.io/en/latest/api/scapy.fields.html#scapy.fields.FCSField

Feel free to reopen the issue if need, or if this reason is not correct.

[kamichal]

In short, the calls to pre_dissect() and post_build() rearrange the packet to place the tail (CRC) field at the end of the packet instead of in the header.

The idea is brilliant. However, the post_build() method is not being called if a Package instance has been created out of raw bytes - because of that if: https://github.com/secdev/scapy/blob/c9d8091335a204050e315ae7de96129b58f45dad/scapy/packet.py#L604-L607

I study the scapy's source code and didn't got any reason for that if yet. In practice whatever you define in post_build there is no chance it will be executed if you created a packet from bytes, e.g. by sniffing. :disappointed:

[matsievskiysv]

In short, the calls to pre_dissect() and post_build() rearrange the packet to place the tail (CRC) field at the end of the packet instead of in the header.

Note that there's an important detail about this approach:

def post_dissect(self, s):
        self.raw_packet_cache = None  # Reset packet to allow post_build
        return s

Otherwise, your packages will be messed up by show2 method (and probably others).