Multiple Problems During Bruteforce

[ghost]

1. Tried running sublist3r using 20 threads on bruteforce and it freezes. Verbose is ON but it's not enumerating any more. I've waited for almost 6 hours but nothing happened.
2. During bruteforce (verbose is ON), multiple copies of the same subdomain are being enumerated. I think this causes the bruteforce method to take some considerable amount of time.
3. Sometimes, running sublist3r prints this error:

Process NetcraftEnum-6: Traceback (most recent call last): File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap self.run() File "sublist3r.py", line 461, in run domain_list = self.enumerate() File "sublist3r.py", line 506, in enumerate cookies = self.create_cookies(resp.headers['set-cookie']) File "/usr/lib/python2.7/dist-packages/requests/structures.py", line 56, in getitem return self._store[key.lower()][1] KeyError: 'set-cookie'

[aboul3la]

Thanks @elvinguitar for reporting these issues.

Regarding issue #3 it's fixed now in commit cb3467a48b6dbbc2e34c3f6c693556e859633671

For issues #1 and #2 i can't see them happening now, so can you show me an example and a test case that demonstrate these issues?

[ghost]

I'm using the latest commit and so far I can't reproduce issues 1 & 2. I'll let you know if I encountered issues 1 & 2 again. By the way, I observed that sometimes when the tool is using Baidu to search for subdomains, it freezes and when I interrupt it using Ctrl+C, it gives the following error:

Yahoo: hangouts.google.com
Baidu: translate.google.com
 ^CProcess BaiduEnum-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "sublist3r.py", line 202, in run
Traceback (most recent call last):
  File "sublist3r.py", line 709, in <module>
    main()
  File "sublist3r.py", line 681, in main
    domain_list = self.enumerate()
    enum.join()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 145, in join
    res = self._popen.wait(timeout)
  File "/usr/lib/python2.7/multiprocessing/forking.py", line 154, in wait
  File "sublist3r.py", line 187, in enumerate
    return self.poll(0)
  File "/usr/lib/python2.7/multiprocessing/forking.py", line 135, in poll
    self.should_sleep()
    pid, sts = os.waitpid(self.pid, flag)
  File "sublist3r.py", line 434, in should_sleep
KeyboardInterrupt
    time.sleep(random.randint(2, 5))
KeyboardInterrupt

[aboul3la]

I'm not sure why Baidu was freezing with you. does it happen with different domains or a specific domain ? and how frequently it occurred ?

[ghost]

It happens on different domains. sometimes it freezes, sometimes not. Now I'm getting this error:

HTTPSConnectionPool(host='dnsdumpster.com', port=443): Read timed out. (read timeout=10)
Process DNSdumpster-7:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "sublist3r.py", line 556, in run
    domain_list = self.enumerate()
  File "sublist3r.py", line 610, in enumerate
    post_resp = self.req('POST', self.base_url, params)
  File "sublist3r.py", line 589, in req
    resp = self.session.post(url, data=params, headers=headers, timeout=self.timeout)
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 518, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 475, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 585, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/adapters.py", line 479, in send
    raise ReadTimeout(e, request=request)
ReadTimeout: HTTPSConnectionPool(host='dnsdumpster.com', port=443): Read timed out. (read timeout=10)

[aboul3la]

Sometimes there is a problem with the dnsdumpster.com website like it could be down or overloaded which cause this problem to occurs,but it's not permanent.

It will work fine if you tried again after few mins.