Problem with DNSdumpster

[ghost]

Got the following error:

root@kali:~/Sublist3r# ./sublist3r.py -t 40 -d site.com

             ____        _     _ _     _   _____
            / ___| _   _| |__ | (_)___| |_|___ / _ __
            \___ \| | | | '_ \| | / __| __| |_ \| '__|
             ___) | |_| | |_) | | \__ \ |_ ___) | |
            |____/ \__,_|_.__/|_|_|___/\__|____/|_|

             # Coded By Ahmed Aboul-Ela - @aboul3la

[-] Enumerating subdomains now for site.com [-] Searching now in Baidu.. [-] Searching now in Yahoo.. [-] Searching now in Google.. [-] Searching now in Bing.. [-] Searching now in Ask.. [-] Searching now in Netcraft.. [-] Searching now in DNSdumpster.. [-] Searching now in Virustotal.. [-] Searching now in ThreatCrowd.. [-] Searching now in SSL Certificates.. [-] Searching now in PassiveDNS.. Process DNSdumpster-8: Traceback (most recent call last): File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap self.run() File "./sublist3r.py", line 565, in run domain_list = self.enumerate() File "./sublist3r.py", line 625, in enumerate token = self.get_csrftoken(resp) File "./sublist3r.py", line 620, in get_csrftoken token = csrf_regex.findall(resp)[0] IndexError: list index out of range

[ghost]

@aboul3la

i get this too in DNSdumpster

Process DNSdumpster-8:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/share/Sublist3r/sublist3r.py", line 565, in run
    domain_list = self.enumerate()
  File "/usr/share/Sublist3r/sublist3r.py", line 625, in enumerate
    token = self.get_csrftoken(resp)
  File "/usr/share/Sublist3r/sublist3r.py", line 620, in get_csrftoken
    token = csrf_regex.findall(resp)[0]
IndexError: list index out of range

[ghost]

i suggest add module that handle connections error

[aboul3la]

Looks like the problem with the DNSdumpster website itself. As the website "https://dnsdumpster.com/" is down right now and it's not working properly. So the issue should be fixed when the website back to the service again.

[aboul3la]

@d4master, I'm gonna add a check for that in the code, so if the website doesn't respond with "200 OK" then it will ignore it and temporary stop the enumeration from that website.

[acaetano]

I'm getting a different error and dnsdumpster is online.

[-] Searching now in PassiveDNS.. HTTPSConnectionPool(host='dnsdumpster.com', port=443): Read timed out. (read timeout=25) Process DNSdumpster-8: Traceback (most recent call last): File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap self.run() File "./sublist3r.py", line 565, in run domain_list = self.enumerate() File "./sublist3r.py", line 628, in enumerate self.extract_domains(post_resp) File "./sublist3r.py", line 641, in extract_domains results_tbl = tbl_regex.findall(resp)[0] TypeError: expected string or buffer

Maybe they changed the output format/syntax?

[gloomy-ghost]

HTTPSConnectionPool(host='dnsdumpster.com', port=443): Read timed out. (read timeout=10)
Exception in thread Thread-7:
Traceback (most recent call last):
  File "%Python%\Python35\lib\threading.py", line 914, in _bootstrap_inner
    self.run()
  File "%Sublist3r%\sublist3r.py", line 258, in run
    domain_list = self.enumerate()
  File "%Sublist3r%\sublist3r.py", line 635, in enumerate
    self.extract_domains(post_resp)
  File "%Sublist3r%\sublist3r.py", line 647, in extract_domains
    results_tbl = tbl_regex.findall(resp)[0]
TypeError: expected string or bytes-like object

[aboul3la]

This bug should be fixed now.

[hoseta]

If any1 will have the same problem just follow my instructions

1. Install beautifulsoup4

pip install beautifulsoup4

2. open sublist3r.py file in any editor.

and import bs4

add this line on the top of file from bs4 import BeautifulSoup

go to line 640 and replace function get_csrftoken

def get_csrftoken(self, resp)

to following

def get_csrftoken(self, resp):

        soup = BeautifulSoup(resp,features="html.parser")

        token = soup.find('input', {'name': 'csrfmiddlewaretoken'}).get('value')

        return token.strip()

[AdriiiPRodri]

Hi everyone,

This problem is still present, the @hoseta fix should be merge in the main branch. It improves the results obtained and does not break the compatibility with Python2.

Before

[-] Total Unique Subdomains Found: 9

After

[-] Total Unique Subdomains Found: 10

[aboul3la]

Hi @hoseta @AdriiiPRodri, thanks for reporting this. This issue was caused due to DNSDumpster was making some modifications in their website HTML structure.

I just pushed a fix for this issue in the latest commit 5d33d9f25a92cc3f91eb351cff2d7007c198fb44, but I will try to apply a better fix using HTML parser better than relying on constant regex values.

[fitzpr]

I'm still getting the error from DNSDumpster. I changed the CSRF code with @aboul3la fix but nothing changed? Maybe I missed something.

Process DNSdumpster-80: Traceback (most recent call last): File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/home/Sublist3r/sublist3r.py", line 271, in run domain_list = self.enumerate() File "/home/Sublist3r/sublist3r.py", line 654, in enumerate self.extract_domains(post_resp) File "/home/Sublist3r/sublist3r.py", line 666, in extract_domains results_tbl = tbl_regex.findall(resp)[0] TypeError: expected string or buffer