search

dotpcap / sharppcap

Official repository - Fully managed, cross platform (Windows, Mac, Linux) .NET library for capturing packets
1.34k stars 268 forks source link

Error opening dump file - 'Too many open files' #8

Closed jwilczek closed 7 years ago

jwilczek commented 8 years ago

I'm attempting to write a tool to split a larger PCAP file into smaller files by host pairs. I'm opening a file and stepping through packet by packet, identifying the host pair, and then writing that packet to the appropriate file.

I'm running into an issue when I get over 508 open files: Error opening dump file 'C:\PCAPOUT\20160623-010218_10.1.129.57_17.172.238.221.pcap: Too many open files'.

It seems like this is an arbitrary limit. I've written similar tools in Python and have had no issues with over 32,000 simultaneous open files. Is this something that can be fixed/modified?

abdullah2993 commented 8 years ago

You are using Windows or Linux?

jwilczek commented 8 years ago

Windows.

abdullah2993 commented 8 years ago

@jwilczek this is not shappcap problem its a windows one. These limits are imposed by the OS. Look here. A possible workaround without tinkering with the OS is to Open/Close with every write or come up with a method to limit the number of simultaneously opened files(post the code/sample pcap).

Kindly close this issue if you think this was the problem /cc @chmorgan

jwilczek commented 8 years ago

I apologize for the slow reply. I'm not sure if I agree that this is can be attributed to an OS issue. First, I experience the error around just 508 open files. The link you pointed me to suggested that 16k was possible. Second, I was able to work around the issue by writing my own PCAP writer. Using System.File.IO.Open(), I was able to have 1000s of simultaneously open files with no issues.

Based on some of the preliminary research I conducted, I was of the opinion that this could be an issue with the underlying winpcap library, which is written in c/c++ (I believe). From my preliminary research, I believe I read that a compile-time flag/switch could be used in c/c++ to increase the number of open files.

Either way, I would disagree that this is a C# limitation.

Sincerely,

Josh

abdullah2993 commented 8 years ago

@jwilczek I may have misread the situation you may be right but IMO this is not a limitation of SharpPcap, could be a wincap one.