search

opnsense / core

OPNsense GUI, API and systems backend
https://opnsense.org/
BSD 2-Clause "Simplified" License
3.36k stars 754 forks source link

Packet Capture stops on HTTP "301 Moved permanently" response #3270

Closed JasMan78 closed 5 years ago

JasMan78 commented 5 years ago

Describe the bug I tried to troubleshoot an issue with my Internet radio. Therefore I started an capture of all packets from its IP address 192.168.10.21. I switched between the radio stations and one of them sends an HTTP "301 Moved permanently" response. When I downloaded the capture file I noticed, that the capture stopped immediatly after the HTTP 301 response.

To Reproduce Steps to reproduce the behavior:

  1. Start "Packet Capture" of your LAN interface
  2. Open HTTP link which response with HTTP 301 code
  3. Open other HTTP links or something else to create more traffic which should appear in the packet capture file
  4. Stop and download capture

Expected behavior The packet capture should not stop after HTTP 301 response

Relevant log files Find attached the capture file from my troubleshooting. I started with the radio station which sends the HTTP 301 response. I switched to two other stations, but the capture did not contain those packets. Also the audio stream is not shown. OPNsense_Capture_Bug.zip

Additional context Because of another reported problem regarding the packet capture and IPS (#3175), I also tried to capture the traffic with IDS/IPS disabled, but without success.

Environment OPNsense 19.1.2 (amd64, OpenSSL).
Intel(R) Celeron(R) CPU J1900 @ 1.99GHz (4 cores) OnBoard Realtek 2 x Realtek GbE OnBoard LAN chips (10/100/1000 Mbit)

AdSchellevis commented 5 years ago

realtek driver issues? is tcpdump still active after your 301? (ps fax | grep tcpdump before stopping the capture) if it crashes for some vague reason, you could try to tcpdump from the console and see what happens

JasMan78 commented 5 years ago

tcpdump is still running and capturing after 301. The size of the downloaded capture file gets bigger when I start a download from another client after the 301 response to my radio. But the download is also not shown in Wireshark.

Wireshark is reporting an issue when I open the capture file: grafik

I didn't read the error message properly before, because I thought it was because of stopping the capture in the middle of the stream. I did an additional capture on my internet ADSL modem and this capture is shown completly in Wireshark. It shows the packets after the 301 response.

That means something is wrong with the OPNsense/tcpdump capture file right after the 301 response.

AdSchellevis commented 5 years ago

looks like a corrupted packet breaking your pcap, you could try to see what tcpdump reports with maximum output enabled.

JasMan78 commented 5 years ago

I did an "tcpdump -vvv" capture. The packet ID 32321 is the one with the 301 response. The following packets look fine in my opinion.

...
12:32:37.728223 00:22:61:21:06:e4 > fc:aa:14:e2:bf:f1, ethertype IPv4 (0x0800), length 154: (tos 0x0, ttl 60, id 30035, offset 0, flags [none], proto TCP (6), length 140)     192.168.10.21.53857 > 144.76.106.52.9090: Flags [P.], cksum 0x8423 (correct), seq 1:101, ack 1, win 31000, length 100
12:32:37.767082 fc:aa:14:e2:bf:f1 > 00:22:61:21:06:e4, ethertype IPv4 (0x0800), length 54: (tos 0x0, ttl 52, id 32320, offset 0, flags [DF], proto TCP (6), length 40)     144.76.106.52.9090 > 192.168.10.21.53857: Flags [.], cksum 0x480b (correct), seq 1, ack 101, win 229, length 0
12:32:37.767409 fc:aa:14:e2:bf:f1 > 00:22:61:21:06:e4, ethertype IPv4 (0x0800), length 469: (tos 0x0, ttl 52, id 32321, offset 0, flags [DF], proto TCP (6), length 455)     144.76.106.52.9090 > 192.168.10.21.53857: Flags [P.], cksum 0xf2db (correct), seq 1:416, ack 101, win 229, length 415
12:32:37.767528 fc:aa:14:e2:bf:f1 > 00:22:61:21:06:e4, ethertype IPv4 (0x0800), length 54: (tos 0x0, ttl 52, id 32322, offset 0, flags [DF], proto TCP (6), length 40)     144.76.106.52.9090 > 192.168.10.21.53857: Flags [F.], cksum 0x466b (correct), seq 416, ack 101, win 229, length 0
12:32:37.771084 00:22:61:21:06:e4 > fc:aa:14:e2:bf:f1, ethertype IPv4 (0x0800), length 56: (tos 0x0, ttl 60, id 30058, offset 0, flags [none], proto TCP (6), length 40)     192.168.10.21.53857 > 144.76.106.52.9090: Flags [.], cksum 0xcfd6 (correct), seq 101, ack 417, win 30585, length 0
12:32:37.771710 00:22:61:21:06:e4 > fc:aa:14:e2:bf:f1, ethertype IPv4 (0x0800), length 56: (tos 0x0, ttl 60, id 30059, offset 0, flags [none], proto TCP (6), length 40)     192.168.10.21.53857 > 144.76.106.52.9090: Flags [.], cksum 0xce37 (correct), seq 101, ack 417, win 31000, length 0
12:32:37.776346 00:22:61:21:06:e4 > fc:aa:14:e2:bf:f1, ethertype IPv4 (0x0800), length 56: (tos 0x0, ttl 60, id 30061, offset 0, flags [none], proto TCP (6), length 40)     192.168.10.21.53857 > 144.76.106.52.9090: Flags [FP.], cksum 0x4746 (correct), seq 101, ack 417, win 1, length 0
12:32:37.778885 00:22:61:21:06:e4 > fc:aa:14:e2:bf:f1, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 60, id 30062, offset 0, flags [none], proto UDP (17), length 60)     192.168.10.21.56973 > 192.168.10.1.53: [udp sum ok] 201+ A? hirschmilch.de. (32)
12:32:37.815112 fc:aa:14:e2:bf:f1 > 00:22:61:21:06:e4, ethertype IPv4 (0x0800), length 54: (tos 0x0, ttl 52, id 57207, offset 0, flags [DF], proto TCP (6), length 40)     144.76.106.52.9090 > 192.168.10.21.53857: Flags [.], cksum 0x466a (correct), seq 417, ack 102, win 229, length 0
12:32:37.841507 fc:aa:14:e2:bf:f1 > 00:22:61:21:06:e4, ethertype IPv4 (0x0800), length 90: (tos 0x0, ttl 64, id 55843, offset 0, flags [none], proto UDP (17), length 76)     192.168.10.1.53 > 192.168.10.21.56973: [udp sum ok] 201 q: A? hirschmilch.de. 1/0/0 hirschmilch.de. A 144.76.106.52 (48)
12:32:37.855019 00:22:61:21:06:e4 > fc:aa:14:e2:bf:f1, ethertype IPv4 (0x0800), length 62: (tos 0x0, ttl 60, id 30107, offset 0, flags [none], proto TCP (6), length 48)     192.168.10.21.53858 > 144.76.106.52.7000: Flags [S], cksum 0x9fe8 (correct), seq 769066086, win 1, options [mss 1460,wscale 0,eol], length 0
...
AdSchellevis commented 5 years ago

I don't think I can really help you out here, if tcpdump creates a corrupt pcap under some specific conditions, it would likely affect standard FreeBSD as well.

You could try to search for some public non https website also serving a 301 and create a scenario we could try out on another machine, although I personally don't expect the 301 is the reason for the corrupted pcap.

JasMan78 commented 5 years ago

I found two other HTTP websites which response with HTTP 301.

The capture of the first site www.cinastic.de is fine. I get no error when I open the file in Wireshark. The packets after the 301 response are shown.

The capture of the second site www.grohde.de has the same issue when I try to open it in Wireshark. The packet list ends right after the HTTP 301 packet.

I also captured the traffic on my DSL modem and compared the 301 responses. The "good" response did not contain the "line-based text data" (HTML) section. The "bad" ones from my Internet radio station and www.grohde.de contains this section.

Can you reproduce this issue on your maschine?

AdSchellevis commented 5 years ago

@JasMan78 I can reproduce it over here, easiest is to try capturing with wireshark directly. It looks like the host (81.88.32.253) sends corrupted traffic, which then creates a defective pcap in tcp dump.

image

We'll leave the issue open as an upstream bug.

JasMan78 commented 5 years ago

My card doesn't show me the FCS. When I capture with Wireshark directly on the host, everything looks fine. Also with Tshark. Thank you for your support.

fichtner commented 5 years ago

Please try https://github.com/opnsense/core/commit/5d4599e7b

# opnsense-patch 5d4599e7b

Suspect the file download was damaged somehow, probably mangled by not opening in binary mode.

JasMan78 commented 5 years ago

@fichtner: I applied the patch and rebooted OPNsense, but it didn't solved the issue.

fichtner commented 5 years ago

Maybe unrelated, it should fix the mentioned issue on the forum, but worth a try here (malformed pcap download as witnessed by the wireshark).

AdSchellevis commented 5 years ago

This issue has been automatically timed-out (after 180 days of inactivity).

For more information about the policies for this repository, please read https://github.com/opnsense/core/blob/master/CONTRIBUTING.md for further details.

If someone wants to step up and work on this issue, just let us know, so we can reopen the issue and assign an owner to it.