pyshark crashes when reading incomplete traces

[marten-seemann]

pyshark throws an exception when reading a pcap that cuts a packet in half. For reference, Wireshark displays the following error message when opening the file: "The capture file appears to have been cut short in the middle of a packet."

The following trace can be used to reproduce the error: trace.pcap.zip (Github doesn't allow me to upload a pcap, so please unzip it).

import pyshark

cap = pyshark.FileCapture("trace.pcap")
for p in cap:
  print(p)

pyshark crashes saying

Traceback (most recent call last):
  File "test.py", line 5, in <module>
    for p in cap:
  File "/usr/local/lib/python3.7/site-packages/pyshark/capture/capture.py", line 259, in _packets_from_tshark_sync
    self.eventloop.run_until_complete(self._cleanup_subprocess(tshark_process))
  File "/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 583, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.7/site-packages/pyshark/capture/capture.py", line 434, in _cleanup_subprocess
    % process.returncode)
pyshark.capture.capture.TSharkCrashException: TShark seems to have crashed (retcode: 2). Try rerunning in debug mode [ capture_obj.set_debug() ] or try updating tshark.
Exception ignored in: <function Capture.__del__ at 0x10650fdd0>
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/pyshark/capture/capture.py", line 446, in __del__
  File "/usr/local/lib/python3.7/site-packages/pyshark/capture/capture.py", line 437, in close
  File "/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 583, in run_until_complete
  File "/usr/local/lib/python3.7/site-packages/pyshark/capture/capture.py", line 441, in close_async
  File "/usr/local/lib/python3.7/site-packages/pyshark/capture/capture.py", line 434, in _cleanup_subprocess
pyshark.capture.capture.TSharkCrashException: TShark seems to have crashed (retcode: 2). Try rerunning in debug mode [ capture_obj.set_debug() ] or try updating tshark.

[marten-seemann]

How was this completed?

[KimiNewt]

It wasn't, just an automatic cleanup. Reopened and will take a look.

[KimiNewt]

It appears to work fine until it reaches the actual cut-off packet. I think that even if we raise a more specific exception, we should still raise one (at least by default), as ignoring cut packets might be more confusing or even not the desired effect. You can see the same error that wireshark produces if you run with debug=True

[mfranzil]

Using Pyshark 0.5.2 built from source, can confirm that issue is still around. My code:

def monitor_pyshark(device):
    cap = pyshark.LiveRingCapture(
        interface=device,
        bpf_filter="udp",
        use_json=True,
        use_ek=True,
        include_raw=True,
        ring_file_size=1024 * 20,
        num_ring_files=3
    )

    try:
        cap.apply_on_packets(packet_callback_pyshark)
    except KeyboardInterrupt:
        exit(0)

Playing with the ring file parameters does not affect the output. When such a setting is used, the last packet of a network flow is systematically held in buffer until more packets arrive.

This is a snippet of a QUIC monitoring application I'm developing. The first screenshot shows what a small flow outputs. The second screenshot shows the same output when using other sniffers (e.g. scapy) or once another network flow arrives with Pyshark. In the first screenshot, the APPLICATION_CLOSE frame is missing.

I've debugged in every possible way, but obtained no relevant information about what is happening about that packet.