search

nmap / npcap

Nmap Project's Windows packet capture and transmission library
https://npcap.com
Other
2.91k stars 509 forks source link

Npcap strange behavior with WireShark #334

Closed jeffrimko closed 3 years ago

jeffrimko commented 4 years ago

Been seeing strange behavior with Wireshark and the Npcap Loopback Adapater. After starting a Wireshark capture on that interface, websites no longer load in a browser. For example, trying to browse to Wikipedia might result in a "Can’t reach this page" in Microsoft Edge or a ERR_CONNECTION_ABORTED/ERR_CONNECTION_RESET in Chrome. This issue will continue after the capture has stopped until I manually stop Npcap in a terminal using "net stop npcap".

It's possible that just having Npcap running is the issue but the issue seems to consistently happen after starting a Wireshark capture.

I added a screen capture of the issue here: https://youtu.be/eUuBBvwRU0g

Windows 10 version 1803 Wireshark version 3.0.5 (v3.0.5-0-g752a55954770) Npcap version 0.9983 DiagReport-20191017-164533.txt

dmiller-nmap commented 4 years ago

Since you have Npcap 0.9983 installed, you no longer need the Npcap Loopback Adapter in order to do loopback capture. This dummy adapter was needed in previous releases, but had an unfortunate side effect of causing routing problems especially on systems with multiple network adapters or VPNs. The only reason it is left in is for legacy support for certain programs that need it, primarily Nmap 7.80 and earlier. Try reinstalling Npcap and un-checking the "Legacy loopback" installation option. If this does not solve the problem, we will see what remains to be done.

jeffrimko commented 4 years ago

Thanks for the response! I tried the following but am still seeing the issue:

A few observations:

dmiller-nmap commented 4 years ago

Yes, Wireshark requires Npcap to be running in order to do capture because Npcap is what does the capturing, so your observations are as expected. I will look into this and see if I can reproduce your issue here.

dmiller-nmap commented 4 years ago

I have not been able to reproduce this issue, so I have a few more diagnostic questions or suggestions:

  1. Is your VPN connected when you experience this issue? If so, does the issue persist if the VPN is not disconnected?
  2. Have you experienced the issue on any other systems, and if so, what do those have in common with this one?
  3. You may have leftover Npcap Loopback adapters from previous installations of Npcap. Remove them according to nmap/npcap#55 and try again.
  4. There may be a conflict or problem with Citrix's Deterministic Network Enhancer (DNE), so removing that or disabling it may fix your problem. Let us know if this is the case.
jeffrimko commented 4 years ago

Answering your questions in order:

  1. The issue shows up both connected and disconnected to the VPN.
  2. Think issue has been seen on another work PC but I can't confirm at the moment. Could be a weird interaction with some company policy software, not sure.
  3. Will do. Very certain there were older versions of Npcap installed on this PC previously.
  4. Okay, I'm not familiar with DNE but will look into it.

Big thanks again and I'll report back with any new info soon!

jeffrimko commented 4 years ago

Okay, some updates related to the previous list of questions:

  1. I didn't find any leftover Npcap Loopback adapters from previous installs.
  2. I'm still not sure what DNE is but suspect it's not a factor here.

The issue is definitely a direct result of the "C:\Windows\System32\Npcap\wpcap.dll" file. When I move this file to a temporary location, the issue goes away and the loopback adapter does not show up in Wireshark. When this file is in its normal location, the issue occurs immediately after starting Wireshark and it loads the list of interfaces (i.e. no need to start a capture).

I did notice some strange Wireshark traffic graphs with npcap enabled and disabled (by moving the wpcap.dll): image

Maybe there is some strange interaction with Hyper-V (this is Windows 10 Pro, I forgot to mention that previously, sorry). I might try to temporarily disable Hyper-V later using these instructions found on StackOverflow. I cannot permanently disable Hyper-V since Docker is needed on this machine.

guyharris commented 4 years ago

If Npcap is disabled in the second screenshot, presumably WinPcap is being used - some capture driver, packet.dll for that driver, and capture DLL for Wireshark to use is present.

jeffrimko commented 4 years ago

Quick update, I haven't investigated this much further recently. After some reading, was scared away from disabling Hyper-V since some people report issues afterwards. If anyone has advice on how to debug this, please give a holler.

daulis commented 4 years ago

I believe that I'm having the same issue with users here as well.

Web browsing stops working after starting Wireshark/Npcap 0.9986

Setup:

  1. Clean system: Uninstall previous Wireshark + Npcap. Windows 10, version 1803.
  2. Install Wireshark-win64-3.2.1.exe (This includes Npcap 0.9986). Default options to use Npcap were used (all boxes, including Legacy loopback adapter were NOT checked)
  3. Confirmed via Device Manager that no previous Loopback adapters were installed.
  4. Web browsing works
  5. Start Wireshark. But, don't start capturing anything.
  6. Web browsing stops working
  7. "net stop npcap" - This allows web browsing to work again

Other notes:

  1. This is not reproducible on all PCs, just certain users.
  2. This issue is present with WinPcap API-compatible mode enabled OR disabled.
  3. Issue seems similar to: https://github.com/nmap/nmap/issues/1173. The workaround for Issue 1173 was to not install the loopback adapter. But, the new issue happens even without installing the Legacy loopback adapter. Is it possible that the new method of capturing loopback traffic (https://github.com/nmap/npcap/releases/tag/v0.9983), now triggers this defect, even though the loopback adapter isn't "installed"?

DiagReport-20200122-135533.txt

dmiller-nmap commented 4 years ago

When Wireshark starts up, it begins packet capture on all interfaces, using the statistics to create the "sparkline" traffic graphs next to each interface. This is likely the event that is triggering the network interruption, which as @daulis points out is likely unavoidable now that the Npcap Loopback Adapter is not required in order to do loopback capture. You may be able to avoid this by using the command-line to start Wireshark, specifying a particular interface with the -i option, which ought to skip the sparkline display. Of course this would only be a workaround; we would rather identify the root cause of the problem and fix it.

I have just a few more questions that I would like to have answered to help me narrow down the scope of the issue:

  1. Does closing Wireshark (and therefore turning off the sparkline graphs and closing their packet capture handles) restore connectivity? Or is stopping the npcap driver really the only way to restore it?
  2. If the above workaround (using Wireshark's -i command line option) successfully avoids interruption, then does interruption happen if you use -i to specify the loopback adapter AND -p to avoid "promiscuous mode"?
daulis commented 4 years ago

I tried these things on a different PC than my previous report, so I can confirm that it's reproducible on at least 2x PCs here. For all of my testing, I used Wireshark 3.2.1. I left Wireshark installed, and only changed Npcap versions for my testing below.

Some extra info that I did before answering your questions above:

Answers to your questions. For these steps, I installed 0.9986, and unchecked "Install Npcap in WinPcap API-compatible Mode". I did this to match what Wireshark sets as defaults during install.

  1. "Does closing Wireshark (and therefore turning off the sparkline graphs and closing their packet capture handles) restore connectivity? Or is stopping the npcap driver really the only way to restore it?" Answer: No, closing Wireshark does not help. Only stopping the npcap service worked. When I started the npcap service after this, web browsing worked (until I started Wireshark again)

  2. "If the above workaround (using Wireshark's -i command line option) successfully avoids interruption, then does interruption happen if you use -i to specify the loopback adapter AND -p to avoid "promiscuous mode"?" Test Results:

    • "Wireshark.exe -i 1" - This still triggered the problem. The "-i" by itself seems to do nothing.
    • "Wireshark.exe -i 1 -k" - This still triggered the problem. The "-k" starts capturing on the selected interface right away
    • "Wireshark.exe -i 1 -k -p" - This still triggered the problem.
    • "dumpcap.exe -D" - This just lists interfaces. This triggered the problem.

It appears just the act of Listing Interfaces is enough to trigger the issue.

guyharris commented 4 years ago

Note that giving -i with a number involves listing interfaces, so that tcpdump/TShark/Wireshark/dumpcap/etc. can translate ordinal numbers into interface names.

You'd have to give the Big Ugly String With a GUID as the -i argument in order to avoid listing interfaces.

daulis commented 4 years ago

@guyharris When I tried using GUID, Wireshark still does the "Finding local interfaces" step. I tried using Wireshark.exe -i \Device\NPF_{EC3585DF-28AE-4F3B-BBFC-7F120F22046D} -k

guyharris commented 4 years ago

Then you'll have to try it with dumpcap -i and the GUID string.

daulis commented 4 years ago

dumpcap -i \Device\NPF_{EC3585DF-28AE-4F3B-BBFC-7F120F22046D} triggers the same bug

guyharris commented 4 years ago

Unfortunately, the code that processes the -i option in all programs in the Wireshark suite calls pcap_findalldevs() to get additional information about the interface (libpcap lacks a "get information about this interface" call; I intend to fix that at some point), so there's no way to avoid pcap_findalldevs() with programs in the Wireshark suite.

This is a job for tcpdump, but there's currently no convenient binary build for recent tcpdump; you might try windump, which may require installation of npcap in WinPcap-compatibility mode.

dmiller-nmap commented 4 years ago

@guyharris Thanks for jumping in with suggestions and explanations. It seems simply opening the Loopback device is sufficient to trigger the bug, so unless @daulis is looking for a workaround that involves separate packet capture with windump and subsequent analysis with Wireshark, it seems the ball is in our court to figure out a fix. We are looking into a separate privately-reported crash related to repeated opening of the loopback adapter, so I'm hopeful we can improve things generally and take care of this bug in the process.

guyharris commented 4 years ago

@guyharris Thanks for jumping in with suggestions and explanations. It seems simply opening the Loopback device is sufficient to trigger the bug,

Or, at least, that any call to pcap_findalldevs() is sufficient to trigger the bug, given that the tests have all been done with programs in the wireshark suite and that, currently, doing any capturing with programs in that suite involves at least one pcap_findalldevs() call.

Given that the pcap_findalldevs() in the current state of the git@github.com:nmap/libpcap.git repository will, on Windows with Npcap (and WinPcap), call PacketOpenAdapter() on each adapter it finds, in order to get interface flags, that means that any use of Wireshark-suite programs to do capture will involve a PacketOpenAdapter() call on every adapter it finds. If it finds a loopback adapter, it'll call PacketOpenAdapter() on it.

Presumably the bug not showing up if there is no loopback adapter is what shows that the bug is probably triggered by opening the loopback adapter.

(Unless 0.9982 has been tested with "Support loopback traffic" checked, it has not been established that the bug was introduced in 0.9983; if, with 0.9982 installed with loopback traffic support enabled, the problem shows up, what has been demonstrated is that 0.9983 removed the one reliable way of preventing the bug from being triggered, at the expense of having loopback capture support removed - which might be a reasonable tradeoff for some users.)

so unless @daulis is looking for a workaround that involves separate packet capture with windump and subsequent analysis with Wireshark,

WinDump will only call pcap_findalldevs() if either 1) the -D flag is specified (the whole point of that flag is to print a list of devices on which you can capture, so calling pcap_findalldevs() is a requirement), 2) an interface name that's just a number is specified (as it has to get the list of devices to turn the ordinal number in that list into an interface name), or 3) no -i flag is specified for a capture (as it has to get the list of devices in order to pick the first device in the list). So a capture with WinDump, using the -i flag with a GUID-based device name, should avoid opening the loopback device and not trigger the bug.

daulis commented 4 years ago

@dmiller-nmap I can pull extra data from one of the PCs that is having this issue, and I can also try out test builds of npcap if that helps you.

dmiller-nmap commented 4 years ago

As a workaround, Npcap 0.9987 released today includes the above commit, which will allow you to use the Windows Registry to disable the loopback WFP callouts, which likely contain the problem code that we have not yet identified. Just set HKLM\SYSTEM\CurrentControlSet\Services\npcap\Parameters\LoopbackSupport to 0 and restart the driver. This is only intended to be a workaround, not a permanent solution, so the Npcap installer does not have a way to set this value, and it will overwrite it with the default value of 1 on reinstall or upgrade. We will continue investigating this issue and hope to find a real fix.

daulis commented 4 years ago

Two observations when using 0.9987:

  1. At one point, when I tried to stop npcap, it failed. After that, it appeared stuck. I had to reboot my machine to recover. I wasn't able to reproduce this. 
    
C:\WINDOWS\system32>net stop npcap
The Npcap Packet Driver (NPCAP) service is stopping........
The Npcap Packet Driver (NPCAP) service could not be stopped.

C:\WINDOWS\system32>net stop npcap The service is starting or stopping. Please try again later.



2. Setting the registry key did work, so it does appear to be caused by the loopback adapter code. (I can't easily use that workaround for my use cases though)
dmiller-nmap commented 4 years ago

This issue should be resolved by Npcap 0.9988, released today. We would appreciate your feedback so we know whether to close this issue.

jeffrimko commented 4 years ago

Just updated to Npcap 0.9988 and the issue appears to be resolved! Thank you greatly!