search

Tib3rius / AutoRecon

AutoRecon is a multi-threaded network reconnaissance tool which performs automated enumeration of services.
GNU General Public License v3.0
5.14k stars 878 forks source link

Secondary scans aren't running properly #110

Closed heinosasshallik closed 3 years ago

heinosasshallik commented 3 years ago

I'm trying to write some custom plugins for AutoRecon, but I noticed that even though I waited ~10 minutes, additional scans (such as gobuster, curl, whatweb, etc) never ran. Here's what I did:

1. Everything works fine by default

First I ran AutoRecon with default settings. Everything works fine.

2. Develop plugin, secondary scans no longer running

I wrote a custom plugin for AutoRecon. Then I executed autorecon and waited for ~10 minutes. Nmap finds services, but the secondary scans never run (except for scans against DNS, for some reason). Here are the logs (this particular scan ran for ~7mins but I've also run it for more):

https://pastebin.com/raw/KwtE5QtD

Here's the plugin I created:

from autorecon.plugins import ServiceScan
from autorecon.io import error, info, fformat
from shutil import which
import os

class DirBusterCustom(ServiceScan):

        def __init__(self):
                super().__init__()
                self.name = "Custom Directory Buster"
                self.slug = 'dirbuster-manual-extensions'
                self.priority = 0
                self.tags = ['custom', 'safe', 'long', 'http']

        def configure(self):
                self.add_choice_option('tool', default='gobuster', choices=['feroxbuster', 'gobuster', 'dirsearch', 'ffuf', 'dirb'], help='The tool to use for directory busting. Default: %(default)s')
                self.add_list_option('wordlist', default=['/home/x90slide/resources/infosec-knowledge/wordlists/web_content/combined_words.txt'], help='The wordlist(s) to use for the custom HTTP scan plugin. Default: %(default)s')

                self.default_threads = 10
                self.default_ext = 'txt,html,php,asp,aspx,jsp'

                self.match_service_name('^http')
                self.match_service_name('^nacn_http$', negative_match=True)

        def check(self):
                tool = self.get_option('tool')
                if tool == 'feroxbuster':
                        if which('feroxbuster') is None:
                                error('The feroxbuster program could not be found. Make sure it is installed. (On Kali, run: sudo apt install feroxbuster)')
                elif tool == 'gobuster':
                        if which('gobuster') is None:
                                error('The gobuster program could not be found. Make sure it is installed. (On Kali, run: sudo apt install gobuster)')
                elif tool == 'dirsearch':
                        if which('dirsearch') is None:
                                error('The dirsearch program could not be found. Make sure it is installed. (On Kali, run: sudo apt install dirsearch)')

        def manual(self, service, plugin_was_run):
                dot_extensions = ','.join(['.' + x for x in self.default_ext.split(',')])
                for wordlist in self.get_option('wordlist'):
                        name = os.path.splitext(os.path.basename(wordlist))[0]
                        if self.get_option('tool') == 'feroxbuster':
                                service.add_manual_command('Enumerate files with extensions manually (change the extensions you want to enumerate).', ['feroxbuster -u {http_scheme}://{addressv6}:{port}/ -t ' + self.default_threads + ' -w ' + wordlist + ' -x "' + self.default_ext + '" -v -k -n -q -o "{scandir}/{protocol}_{port}_{http_scheme}_feroxbuster_' + name + '.txt"'])
                        elif self.get_option('tool') == 'gobuster':
                                service.add_manual_command('Enumerate files with extensions manually (change the extensions you want to enumerate).', ['gobuster dir -u {http_scheme}://{addressv6}:{port}/ -t ' + self.default_threads + ' -w ' + wordlist + ' -e -k -x "' + self.default_ext + '" -z -o "{scandir}/{protocol}_{port}_{http_scheme}_gobuster_' + name + '.txt"'])
                        elif self.get_option('tool') == 'dirsearch':
                                if service.target.ipversion == 'IPv6':
                                        error('dirsearch does not support IPv6.')
                                else:
                                        service.add_manual_command('Enumerate files with extensions manually (change the extensions you want to enumerate).', ['dirsearch -u {http_scheme}://{address}:{port}/ -t ' + self.default_threads + ' -e "' + self.default_ext + '" -f -q -w ' + wordlist + ' --format=plain -o "{scandir}/{protocol}_{port}_{http_scheme}_dirsearch_' + name + '.txt"'])
                        elif self.get_option('tool') == 'ffuf':
                                service.add_manual_command('Enumerate files with extensions manually (change the extensions you want to enumerate).', ['ffuf -u {http_scheme}://{addressv6}:{port}/FUZZ -t ' + self.default_threads + ' -w ' + wordlist + ' -e "' + dot_extensions + '" -v -noninteractive | tee {scandir}/{protocol}_{port}_{http_scheme}_ffuf_' + name + '.txt'])
                        elif self.get_option('tool') == 'dirb':
                                service.add_manual_command('Enumerate files with extensions manually (change the extensions you want to enumerate).', ['dirb {http_scheme}://{addressv6}:{port}/ ' + wordlist + ' -l -r -S -X ",' + dot_extensions + '" -o "{scandir}/{protocol}_{port}_{http_scheme}_dirb_' + name + '.txt"'])

class GobusterCommon(ServiceScan):
        def __init__(self):
                super().__init__()
                self.name = "Gobuster using common.txt with file extensions"
                self.slug = 'gobuster-common'
                self.priority = 0
                self.tags = ['custom', 'safe', 'long', 'http']

        def configure(self):
                self.default_threads = 10
                self.default_ext = 'txt,html,php,asp,aspx,jsp'

                self.match_service_name('^http')
                self.match_service_name('^nacn_http$', negative_match=True)

        def check(self):
                tool = 'gobuster'
                if which('gobuster') is None:
                        error('The gobuster program could not be found. Make sure it is installed. (On Kali, run: sudo apt install gobuster)')

        async def run(self, service):
                wordlist = "/usr/share/seclists/Discovery/Web-Content/common.txt"
                name = os.path.splitext(os.path.basename(wordlist))[0]
                await service.execute('gobuster dir -u {http_scheme}://{addressv6}:{port}/ -t ' + self.default_threads + ' -w ' + wordlist + ' -e -k -x "' + self.default_ext + '" -z -o "{scandir}/{protocol}_{port}_{http_scheme}_gobuster_' + name + '.txt"')

I softlinked the plugin to root's configuration folder like so:

sudo ln -s /home/x90slide/resources/infosec-knowledge/scripts/autorecon_profiles/http_server_custom.py  /root/.config/AutoRecon/plugins/http_server_custom.py

It seems like no matter how long I wait, only the DNS secondary scans are run for some reason.

Run on pure setup to check

I deleted the softlink and ran autorecon again. After ~3mins of waiting, the secondary scans started running (please ignore the first line):

https://pastebin.com/raw/ViQfd0cf

The problem

I'm not really sure what's going wrong here. Did I do something wrong in my plugin? Am I not supposed to softlink plugins to that directory?

Tib3rius commented 3 years ago

Hi @heinosasshallik,

The thing I noticed immediately was that you don't have the "default" tag defined in self.tags. If you're running AutoRecon without specifying any tags, only plugins tagged with "default" will get run.

I believe that should fix your issue, but let me know if it does not.

Alternatively you could also add --tags=default,custom to the command, which should then run all plugins tagged with either default or custom.

heinosasshallik commented 3 years ago

Thanks for spotting that mistake. I've changed the custom tag to default now, but that hasn't fixed it.

As long as my custom plugin is in that folder, none of the other plugins (except for the DNS one) are run.

Here's the scan results with my plugin in the folder (I tried both symlinking and copying, the result is the same):

[*] Scanning target intelligence
[*] Port scan Top TCP Ports (top-tcp-ports) running against intelligence
[*] Port scan All TCP Ports (all-tcp-ports) running against intelligence
[*] Port scan Top 100 UDP Ports (top-100-udp-ports) running against intelligence
[*] Discovered open port udp/53 on intelligence
[*] Discovered open port tcp/139 on intelligence
[*] Discovered open port tcp/53 on intelligence
[*] Discovered open port tcp/80 on intelligence
[*] Discovered open port tcp/135 on intelligence
[*] Discovered open port tcp/445 on intelligence
[*] Discovered open port tcp/49694 on intelligence
[*] Discovered open port udp/123 on intelligence
[*] Discovered open port tcp/3269 on intelligence
[*] Discovered open port tcp/49693 on intelligence
[*] Discovered open port udp/88 on intelligence
[*] Discovered open port tcp/9389 on intelligence
[*] Discovered open port tcp/49704 on intelligence
[*] 17:06:21 - There are 3 scans still running against intelligence: top-tcp-ports, all-tcp-ports, top-100-udp-ports
[*] Discovered open port tcp/636 on intelligence
[*] Discovered open port tcp/389 on intelligence
[*] Discovered open port tcp/464 on intelligence
[*] Discovered open port tcp/5985 on intelligence
[*] Discovered open port tcp/62560 on intelligence
[*] Discovered open port tcp/49716 on intelligence
[*] Discovered open port tcp/593 on intelligence
[*] Discovered open port tcp/88 on intelligence
[*] Discovered open port tcp/49667 on intelligence
[*] Discovered open port tcp/3268 on intelligence
[*] 17:07:21 - There are 3 scans still running against intelligence: top-tcp-ports, all-tcp-ports, top-100-udp-ports
[*] 17:08:21 - There are 3 scans still running against intelligence: top-tcp-ports, all-tcp-ports, top-100-udp-ports
[*] Identified service domain on tcp/53 on intelligence
[*] Identified service http on tcp/80 on intelligence
[*] Service scan DNS Reverse Lookup (tcp/53/domain/dns-reverse-lookup) running against intelligence
[*] Service scan DNS Zone Transfer (tcp/53/domain/dns-zone-transfer) running against intelligence
[*] Service scan Nmap DNS (tcp/53/domain/nmap-dns) running against intelligence
[*] 17:09:21 - There are 3 scans still running against intelligence: top-tcp-ports, top-100-udp-ports, tcp/53/domain/nmap-dns
[*] 17:10:21 - There are 3 scans still running against intelligence: top-tcp-ports, top-100-udp-ports, tcp/53/domain/nmap-dns
[*] 17:11:21 - There are 3 scans still running against intelligence: top-tcp-ports, top-100-udp-ports, tcp/53/domain/nmap-dns

And you can see that the result is different if my plugin isn't in the folder (the other plugins are run, as they should be):


[*] Scanning target intelligence
[*] Port scan Top TCP Ports (top-tcp-ports) running against intelligence
[*] Port scan All TCP Ports (all-tcp-ports) running against intelligence
[*] Port scan Top 100 UDP Ports (top-100-udp-ports) running against intelligence
[*] Discovered open port tcp/139 on intelligence
[*] Discovered open port tcp/135 on intelligence
[*] Discovered open port tcp/53 on intelligence
[*] Discovered open port tcp/80 on intelligence
[*] Discovered open port tcp/445 on intelligence
[*] Discovered open port tcp/49704 on intelligence
[*] Discovered open port tcp/5985 on intelligence
[*] Discovered open port udp/123 on intelligence
[*] Discovered open port tcp/9389 on intelligence
[*] Discovered open port udp/53 on intelligence
[*] Discovered open port udp/88 on intelligence
[*] Discovered open port tcp/49694 on intelligence
[*] Discovered open port tcp/49667 on intelligence
[*] Discovered open port tcp/49693 on intelligence
[*] Discovered open port tcp/62560 on intelligence
[*] Discovered open port tcp/49716 on intelligence
[*] Discovered open port tcp/593 on intelligence
[*] 17:13:05 - There are 3 scans still running against intelligence: top-tcp-ports, all-tcp-ports, top-100-udp-ports
[*] Discovered open port tcp/3268 on intelligence
[*] Discovered open port tcp/389 on intelligence
[*] Discovered open port tcp/464 on intelligence
[*] Discovered open port tcp/3269 on intelligence
[*] Discovered open port tcp/636 on intelligence
[*] Discovered open port tcp/88 on intelligence
[*] 17:14:05 - There are 3 scans still running against intelligence: top-tcp-ports, all-tcp-ports, top-100-udp-ports
[*] 17:15:05 - There are 3 scans still running against intelligence: top-tcp-ports, all-tcp-ports, top-100-udp-ports
[*] Identified service domain on tcp/53 on intelligence
[*] Identified service http on tcp/80 on intelligence
[*] Identified service kerberos-sec on tcp/88 on intelligence
[*] Identified service msrpc on tcp/135 on intelligence
[*] Identified service netbios-ssn on tcp/139 on intelligence
[*] Identified service ldap on tcp/389 on intelligence
[*] Identified service microsoft-ds on tcp/445 on intelligence
[*] Identified service kpasswd5 on tcp/464 on intelligence
[*] Identified service ncacn_http on tcp/593 on intelligence
[*] Identified service ldap on tcp/636 on intelligence
[*] Identified service ldap on tcp/3268 on intelligence
[*] Identified service ldap on tcp/3269 on intelligence
[*] Identified service http on tcp/5985 on intelligence
[*] Identified service mc-nmf on tcp/9389 on intelligence
[*] Identified service msrpc on tcp/49667 on intelligence
[*] Identified service ncacn_http on tcp/49693 on intelligence
[*] Identified service msrpc on tcp/49694 on intelligence
[*] Identified service msrpc on tcp/49704 on intelligence
[*] Identified service msrpc on tcp/49716 on intelligence
[*] Identified service msrpc on tcp/62560 on intelligence
[*] Service scan DNS Reverse Lookup (tcp/53/domain/dns-reverse-lookup) running against intelligence
[*] Service scan DNS Zone Transfer (tcp/53/domain/dns-zone-transfer) running against intelligence
[*] Service scan Nmap DNS (tcp/53/domain/nmap-dns) running against intelligence
[*] Service scan Directory Buster (tcp/80/http/dirbuster) running against intelligence
[*] Service scan Curl (tcp/80/http/curl) running against intelligence
[*] Service scan Curl Robots (tcp/80/http/curl-robots) running against intelligence
[*] Service scan Nmap HTTP (tcp/80/http/nmap-http) running against intelligence
[*] Service scan whatweb (tcp/80/http/whatweb) running against intelligence
[*] Service scan wkhtmltoimage (tcp/80/http/wkhtmltoimage) running against intelligence
[*] Service scan Nmap Kerberos (tcp/88/kerberos-sec/nmap-kerberos) running against intelligence
[*] Service scan Nmap MSRPC (tcp/135/msrpc/nmap-msrpc) running against intelligence
[*] Service scan rpcdump (tcp/135/msrpc/rpcdump) running against intelligence
[*] Service scan Enum4Linux (tcp/139/netbios-ssn/enum4linux) running against intelligence
[*] Service scan nbtscan (tcp/139/netbios-ssn/nbtscan) running against intelligence
[*] Service scan Nmap SMB (tcp/139/netbios-ssn/nmap-smb) running against intelligence
[*] Service scan SMBClient (tcp/139/netbios-ssn/smbclient) running against intelligence
[*] Service scan SMBMap (tcp/139/netbios-ssn/smbmap) running against intelligence
[*] Service scan Nmap LDAP (tcp/389/ldap/nmap-ldap) running against intelligence
[*] Service scan Nmap SMB (tcp/445/microsoft-ds/nmap-smb) running against intelligence
[*] Service scan SMBMap (tcp/445/microsoft-ds/smbmap) running against intelligence
[*] Service scan Nmap Kerberos (tcp/464/kpasswd5/nmap-kerberos) running against intelligence
[*] Service scan Nmap LDAP (tcp/636/ldap/nmap-ldap) running against intelligence
[*] Service scan SSL Scan (tcp/636/ldap/ssl-scan) running against intelligence
[*] Service scan Nmap LDAP (tcp/3268/ldap/nmap-ldap) running against intelligence
[*] Service scan Nmap LDAP (tcp/3269/ldap/nmap-ldap) running against intelligence
[*] Service scan SSL Scan (tcp/3269/ldap/ssl-scan) running against intelligence
[*] Service scan Directory Buster (tcp/5985/http/dirbuster) running against intelligence
[*] Service scan Curl (tcp/5985/http/curl) running against intelligence
[*] Service scan Curl Robots (tcp/5985/http/curl-robots) running against intelligence
[*] Service scan Nmap HTTP (tcp/5985/http/nmap-http) running against intelligence
[*] Service scan whatweb (tcp/5985/http/whatweb) running against intelligence
[*] Service scan wkhtmltoimage (tcp/5985/http/wkhtmltoimage) running against intelligence
[*] Service scan WinRM Detection (tcp/5985/http/winrm-detection) running against intelligence
[*] Service scan Nmap MSRPC (tcp/49667/msrpc/nmap-msrpc) running against intelligence
[*] Service scan rpcdump (tcp/49667/msrpc/rpcdump) running against intelligence
[*] Service scan Nmap MSRPC (tcp/49694/msrpc/nmap-msrpc) running against intelligence
[*] Service scan rpcdump (tcp/49694/msrpc/rpcdump) running against intelligence
[*] Service scan Nmap MSRPC (tcp/49704/msrpc/nmap-msrpc) running against intelligence
[*] Service scan rpcdump (tcp/49704/msrpc/rpcdump) running against intelligence
[*] Service scan Nmap MSRPC (tcp/49716/msrpc/nmap-msrpc) running against intelligence
[*] Service scan rpcdump (tcp/49716/msrpc/rpcdump) running against intelligence
[*] Service scan Nmap MSRPC (tcp/62560/msrpc/nmap-msrpc) running against intelligence
[*] Service scan rpcdump (tcp/62560/msrpc/rpcdump) running against intelligence
[*] [tcp/80/http/curl-robots] There did not appear to be a robots.txt file in the webroot (/).
[*] [tcp/5985/http/curl-robots] There did not appear to be a robots.txt file in the webroot (/).
[*] 17:16:05 - There are 18 scans still running against intelligence: top-tcp-ports, top-100-udp-ports, tcp/53/domain/nmap-dns, tcp/80/http/dirbuster, tcp/80/http/nmap-http, tcp/139/netbios-ssn/nmap-smb, tcp/445/microsoft-ds/nmap-smb, tcp/636/ldap/nmap-ldap, tcp/636/ldap/ssl-scan, tcp/3269/ldap/nmap-ldap, tcp/3269/ldap/ssl-scan, tcp/5985/http/dirbuster, tcp/5985/http/nmap-http, tcp/49667/msrpc/nmap-msrpc, tcp/49694/msrpc/nmap-msrpc, tcp/49704/msrpc/nmap-msrpc, tcp/49716/msrpc/nmap-msrpc, tcp/62560/msrpc/nmap-msrpc
[*] 17:17:05 - There are 9 scans still running against intelligence: top-tcp-ports, top-100-udp-ports, tcp/53/domain/nmap-dns, tcp/80/http/dirbuster, tcp/80/http/nmap-http, tcp/636/ldap/ssl-scan, tcp/3269/ldap/ssl-scan, tcp/5985/http/dirbuster, tcp/5985/http/nmap-http
[*] 17:18:05 - There are 6 scans still running against intelligence: top-tcp-ports, top-100-udp-ports, tcp/53/domain/nmap-dns, tcp/80/http/dirbuster, tcp/5985/http/dirbuster, tcp/5985/http/nmap-http
heinosasshallik commented 3 years ago

Aah, here's something interesting. When I used CTRL+C to end the scan, I got this error message:

[*] 17:43:26 - There is 1 scan still running against intelligence: top-100-udp-ports
Task exception was never retrieved
future: <Task finished name='Task-2' coro=<scan_target() done, defined at /usr/local/lib/python3.9/dist-packages/autorecon/main.py:366> exception=TypeError('can only concatenate str (not "int") to str')>
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/autorecon/main.py", line 630, in scan_target
    plugin.manual(service, plugin_was_run)
  File "/root/.config/AutoRecon/plugins/http_server_custom.py", line 44, in manual
    service.add_manual_command('Enumerate files with extensions manually (change the extensions you want to enumerate).', ['gobuster dir -u {http_scheme}://{addressv6}:{port}/ -t ' + self.default_threads + ' -w ' + wordlist + ' -e -k -x "' + self.default_ext + '" -z -o "{scandir}/{protocol}_{port}_{http_scheme}_gobuster_' + name + '.txt"'])
TypeError: can only concatenate str (not "int") to str

It looks like I've made a programming error in my plugin. But it's a shame that AutoRecon doesn't complain about it during the scan (and even worse, the other plugins break too).

heinosasshallik commented 3 years ago

I've fixed the programming error and everything works now. But I think the real issue here is that AutoRecon fails silently and badly when a plugin is faulty. Creating another issue about this.