OSError: Too many open files

shamrocksu88 commented 1 year ago

What is the current behavior?

Errors out during the scan with error - OSError: [Errno 24] Too many open files when ran with domain list as param.

python3 dirsearch.py -l domains.txt

Tried with various thread values as well but still the same error.

What is the expected behavior?

Should not error out instead directory scan should complete.

Any additional information?

Version: dirsearch v0.4.2 OS: macOS

Error:

[########### ] 56% 6427/11460 161/s job:89/1067 errors:220Traceback (most recent call last): [ ] 0% 5/11460 161/s job:90/1066 errors:220 File "dirsearch.py", line 71, in [ ] 0% 5/11460 161/s job:90/1066 errors:220 File "dirsearch.py", line 67, in main File "/Users/test/Desktop/tool/dirsearch/lib/controller/controller.py", line 79, in init [ ] 0% 6/11460 161/s job:90/1066 errors:220 File "/Users/test/Desktop/tool/dirsearch/lib/controller/controller.py", line 221, in run [ ] 0% 6/11460 161/s job:90/1066 errors:220 File "/Users/test/Desktop/tool/dirsearch/lib/controller/controller.py", line 265, in start File "/Users/test/Desktop/tool/dirsearch/lib/controller/controller.py", line 550, in process [ ] 0% 6/11460 161/s job:90/1066 errors:220 File "/Users/test/Desktop/tool/dirsearch/lib/core/fuzzer.py", line 54, in wait [ ] 0% 7/11460 161/s job:90/1066 errors:220 File "/Users/test/Desktop/tool/dirsearch/lib/core/fuzzer.py", line 184, in scan [ ] 0% 7/11460 161/s job:90/1066 errors:220 File "/Users/test/Desktop/tool/dirsearch/lib/controller/controller.py", line 453, in match_callback [ ] 0% 7/11460 161/s job:90/1066 errors:220 File "/Users/test/Desktop/tool/dirsearch/lib/core/decorators.py", line 57, in with_locking [ ] 0% 8/11460 161/s job:90/1066 errors:220 File "/Users/test/Desktop/tool/dirsearch/lib/reports/base.py", line 37, in save OSError: [Errno 24] Too many open files: '/Users/test/Desktop/tool/dirsearch/reports/BATCH-22-10-06_15-10-27/BATCH.txt'

shelld3v commented 1 year ago

python3 dirsearch.py --version output?

iamxhunt3r commented 1 year ago

dirsearch v0.4.3

maurosoria commented 6 months ago

Please, try one of this bugfixes. If you can try both and give us feedback, far far better!

Increasing File Limit with `ulimit` on macOS

Temporary Increase

Open the Terminal.
Execute ulimit -n 4096 (or another higher number) to increase the file limit temporarily.
Run dirsearch in the same Terminal session.

Permanent Increase

Open your shell configuration file (.bashrc, .bash_profile, .zshrc, etc.) in a text editor.
Add the line ulimit -n 4096 at the end of the file.
Save the file and restart the Terminal.

Modifying `dirsearch` Main Function in Python

Edit the Python Script

Locate the dirsearch.py file.
Open it in a text editor.

Import the `resource` Module

Add import resource at the beginning of the script.

Set File Limit in Python

Add these lines at the beginning of the main function in dirsearch.py:


soft, hard = resource.getrlimit(resource.RLIMIT_NOFILE)
resource.setrlimit(resource.RLIMIT_NOFILE, (4096, hard))

shelld3v commented 6 months ago

@maurosoria I'm not sure but maybe we forgot to close files somewhere? I have a local work that will refactor the entire report-saving thing so let's hope that it will somehow fix this 🙏

maurosoria commented 6 months ago

yes @shelld3v that must be the issue!! Report manager should close the files but for some reason is not doing it. Perhaps a line missing in controller.py ? Does ReportManager has a proper destructor function?

maurosoria / dirsearch