Open GhostDog98 opened 4 hours ago
Also I should note that using the fork method is no longer the default as noted in the multiprocessing page:
Update, here's some minimal reproductions: freeze support issue:
from multiprocessing import Pool
from bbot.scanner import Scanner
def do_things_with_bbot(domain):
print("Starting scan...")
scan = Scanner(domain, presets=["subdomain-enum"])
for result in scan.start():
print(result)
pool = Pool(1) # Pool of size 1
pool.map(do_things_with_bbot, ["example.com"])
Infinite recursion issue:
from multiprocessing import Pool
import multiprocessing as mp
from bbot.scanner import Scanner
mp.set_start_method("fork", force=True)
def do_things_with_bbot(domain):
print("Starting scan...")
scan = Scanner(domain, presets=["subdomain-enum"])
for result in scan.start():
print(result)
pool = Pool(1) # Pool of size 1
pool.map(do_things_with_bbot, ["example.com"])
@GhostDog98 thanks for the detailed report.
This is a known quirk, and exists because BBOT inherently does a heavy amount of parallelization, including spawning its own process pool. Running BBOT in a pool is highly discouraged, as there are better ways to parallelize it, such as turning up the number of threads for the dns engine, modules, etc. Also, since BBOT has global state, running multiple BBOT scans within the same python environment is not recommended.
However, if you really want to do it, you might be able to get it to work by checking the name of the process. This is how BBOT handles it internally:
import multiprocessing
current_process = multiprocessing.current_process()
if current_process.name == "MainProcess":
# do stuff
Describe the bug If we run the sample code provided in the advanced usage page:
In python, we get the correct output. However, if we are running inside of python, in a Pool (even a pool of 1 item, and you have to use the
fork
start method or it does not work!), this causes an infinite loop and error whenever you callscan.start()
The fork issue means you need to do:
mp.set_start_method("fork", force=True)
at the start of files, if:__main__
test case.Producing this error:
Expected behavior BBot should run without issue under a multiprocessing pool with default settings.
BBOT Command See above
OS, BBOT Installation Method + Version OS: Kali linux on WSL BBOT Version: 2.0.1 Python version: 3.11.8 Install method:
pip install bbot==2.0.1