Wfuzz is using an absurd amount of memory when using large wordlists (memory leak?)

[junzhengca]

Issue template

Context

Please check:

• [x] I've read the docs for Wfuzz

Please describe your local environment:

Wfuzz version: 3.1.0

Python version: 3.9.10

OS: Kali Rolling (virtualized in Parallels)

Report

What is the current behavior?

wfuzz is using multiple gigs of memory when using large wordlists

And as you can see from the graph, the memory usage just keeps increasing, so I highly suspect there is a memory leak.

What is the expected or desired behavior?

Should not use so much memory.

Please provide steps to reproduce, including exact wfuzz command executed and output:

wfuzz -c -z file,/home/alice/SecLists/Discovery/Web-Content/combined_directories.txt -t 5 --hc 404,403 https://SOME_TARGET_URL/FUZZ

Unfortunately I cannot provide the exact target domain, but you get the idea.

combined_directories.txt is about 16 MB in size.

Other relevant information:

Running in Parallels on a Mac.

[cyberpathogen2018]

I'm experiencing this too. Bruteforcing a web app authentication with about 9 million entries, wfuzz has balooned to over 5 GB after only 18000 attempts. wfuzz eventually crashes when it maxes out the VM memory (8 GB, after about 30000 requests). If i'm running it in tmux, it takes the tmux server with it.

Please describe your local environment: Wfuzz version: 3.1.0 Python version: 3.10.8 OS: Kali Rolling (in vmware workstation 15)

Command line: wfuzz -t 40 -c -w over8chars.txt --sc 200 -d '{"username":"test","password":"FUZZ"}' -H "Content-Type: application/json" http://api.testdomain/auth/login

[TurboWindX]

@cyberpathogen2018, @junzhengca I found a quick fix that works with systemd-based system (like kali):

systemd-run --scope -p MemoryLimit=6G wfuzz -u example.com/FUZZ -w biglist.txt

Note that this is nowhere near optimal as this will limit the RAM usage to 6GB and will use SWAP if needed/available to keep on running without crashing. Try bumping the MemoryLimit as high as possible to have the best possible speed, I use htop to see how much available memory I have left and I leave myself 1-2GB of room.

It would probably be easier to use another software but I like wfuzz and I'm used to it. I used ffuf for a bit but VHOST fuzzing isn't implemented correctly (target URL automatically changes depending on Host header) so I switched back to wfuzz which is slower but more reliable.