codingo / Interlace

Easily turn single threaded command line applications into a fast, multi-threaded application with CIDR and glob support.
GNU General Public License v3.0
1.19k stars 178 forks source link

Threads have to be same number as hosts #76

Closed MantisSTS closed 4 years ago

MantisSTS commented 5 years ago

So I have just setup a recon "rule" for some bug bounties and found that if I have a lot of hosts (1368) Interlace won't run unless I give it the same number of threads as hosts.

Obviously, this shouldn't be the case and it should just launch however many threads it's given and move on from there.

Result: Hangs indefinitely

interlace -cL ~/Results/Interlace_Rules/Recon.rule -tL /tmp/hosts.txt -threads 30 -o ~/Bounties/TARGET/

Result: Hangs Runs fine

interlace -cL ~/Results/Interlace_Rules/Recon.rule -tL /tmp/hosts.txt -threads 1368 -o ~/Bounties/TARGET/

Hosts file:

root@6c1cafa4143b:~# wc -l /tmp/hosts.txt
1368 /tmp/hosts.txt

The hosts file consists of IP addresses and domain names.

This is the rule I set up:

root@6c1cafa4143b:~# cat ~/Results/Interlace_Rules/Recon.rule
mkdir -p _output_/_target_/scans
_blocker_
/opt/Tools/Recon/Discovery/httprobe/httprobe -c 200 _target_ -p80,8000,8080,8081,8443,443 | tee -a _output_/_target_/scans/http_hosts.txt
_blocker_
cat _output_/_target_/scans/http_hosts.txt | xargs -I{} python3 /root/tools/Recon/Discovery/dirsearch/dirsearch.py -u {} -e php,asp,aspx,jsp,do,htm,html,shtml,jhtml,txt -t 50 -R 2 -r --suppress-empty --random-agents --timeout=3 -b --max-retries 2  --simple-report=_output_/_target_/scans/dirsearch-_target_.out -w /opt/Tools/Wordlists/SecLists/Discovery/Web-Content/raft-medium-directories.txt
nmap -p80,8000,8080,8081,8443,443,21,22,25,3389,3306 -A -vvv -oN _output_/_target_/scans/nmap-_target_-results.txt
root@6c1cafa4143b:~#

Nothing too much in there but it was more of a tester.

Any help on this would be great.

codingo commented 5 years ago

I'm travelling but likely related to #75

codingo commented 5 years ago

@richard-clifford I can look at this properly for you next week, I believe it's going to be solved as we look into #75 deeper

iamOgunyinka commented 5 years ago

I tried to replicate this bug as described above but my research led me to a different conclusion, which may or may not be the same as what you found out. This bug has little to do with the -thread option and more to do with the command preprocessing. I'll be issuing a PR in few minutes.

Done! PR issued here

codingo commented 4 years ago

Closed in #73