AutoRecon failing silently when a plugin is broken

[heinosasshallik]

This issue is related to https://github.com/Tib3rius/AutoRecon/issues/110.

When a plugin gives an exception, then AutoRecon fails silently:

1. It doesn't display any error message a. It did display an exception once, when I pressed CTRL+C. But that didn't happen every time.
2. Other plugins will fail as well

This makes plugin development and AutoRecon customization pretty difficult.

If a plugin is broken and gives an exception, then AutoRecon should show that exception to you (if not by default, then at least when the -v flag is specified). Also it's weird that all the other plugins (except for the DNS one) also fail when there's a faulty plugin.

[Tib3rius]

Thanks for the report, I'll look into it. I know I coded some exception detection into the plugin system so I'll investigate why it isn't firing automatically.

[Tib3rius]

@heinosasshallik I believe this is fixed in 0beb2ad. Can you confirm?

[heinosasshallik]

I ran this command:

sudo python3 -m pip install git+https://github.com/Tib3rius/AutoRecon.git

Then broke my script again and ran the same command I ran before. The result is the same. Only the DNS plugin is being run, and no error is displayed.

No changes observed, problem not fixed.

[Tib3rius]

Can you run:

sudo autorecon --version

and let me know the output? I just want to make sure it actually updated. It should show "AutoRecon v2.0.2".

[heinosasshallik]

Sure. It says "AutoRecon v2.0.1".

[Tib3rius]

@heinosasshallik so yeah, you aren't running the version with the fix. :)

Can you try running:

sudo python3 -m pip install --upgrade git+https://github.com/Tib3rius/AutoRecon.git

Then re-run with --version. Once you have v2.0.2 you can re-run your faulty code and it should catch the exception.

[heinosasshallik]

I ran that command, then ran autorecon --version again. It's still showing version 2.0.1. I also tried closing the terminal and opening it back up, and the result is the same. From running which autorecon, it says it's in the /usr/local/bin directory. Not sure why I'm not getting version 2.0.2.

I didn't originally install it with pipx , either (since pipx isn't installed on my machine).

[Tib3rius]

That's really odd. The latest version of the code in the repo is definitely version 2.0.2: https://github.com/Tib3rius/AutoRecon/blob/0beb2ad7c12c51c88e255f2af7ce9f466f59e673/autorecon/main.py#L20

pip should upgrade by downloading the repo again. Can you try a fresh install (i.e. pip uninstall autorecon) and see if installing it again works?

[heinosasshallik]

Uninstalled, reinstalled. Last messsage by pip is Successfully installed autorecon-2.0.1

[Tib3rius]

Ah, I forgot to update the pyproject.toml file! Can you try again and see if it updates now? Apologies for this, I'm pretty new to creating pip/pipx packages!

[heinosasshallik]

Nope, still the same issue. Sorry about just sending a screenshot instead of the full logs. I don't have clipboard copy enabled between my kali VM and main host, so it's a bit annoying to transfer logs back and forth. Let me know if you need the whole thing. You can see from the screenshot though that a lot of ports were found, but no error was thrown and only port53 was scanned further

[Tib3rius]

Can you share your plugin code again? It looks like this is a different exception, caused by a KeyError in the formatter, but I don't see the "scheme" key being used in the plugin code you pasted the other day.

[heinosasshallik]

Sure. Here it is:

from autorecon.plugins import ServiceScan
from autorecon.io import error, info, fformat
from shutil import which
import os

class DirBusterCustom(ServiceScan):

        def __init__(self):
                super().__init__()
                self.name = "Custom Directory Buster"
                self.slug = 'dirbuster-manual-extensions'
                self.priority = 0
                self.tags = ['default', 'safe', 'long', 'http']

        def configure(self):
                self.add_choice_option('tool', default='gobuster', choices=['feroxbuster', 'gobuster', 'dirsearch', 'ffuf', 'dirb'], help='The tool to use for directory busting. Default: %(default)s')
                self.add_list_option('wordlist', default=['/home/x90slide/resources/infosec-knowledge/wordlists/web_content/combined_words.txt'], help='The wordlist(s) to use for the custom HTTP scan plugin. Default: %(default)s')

                self.default_threads = 10
                self.default_ext = 'txt,html,php,asp,aspx,jsp'

                self.match_service_name('^http')
                self.match_service_name('^nacn_http$', negative_match=True)

        def check(self):
                tool = self.get_option('tool')
                if tool == 'feroxbuster':
                        if which('feroxbuster') is None:
                                error('The feroxbuster program could not be found. Make sure it is installed. (On Kali, run: sudo apt install feroxbuster)')
                elif tool == 'gobuster':
                        if which('gobuster') is None:
                                error('The gobuster program could not be found. Make sure it is installed. (On Kali, run: sudo apt install gobuster)')
                elif tool == 'dirsearch':
                        if which('dirsearch') is None:
                                error('The dirsearch program could not be found. Make sure it is installed. (On Kali, run: sudo apt install dirsearch)')

        def manual(self, service, plugin_was_run):
                dot_extensions = ','.join(['.' + x for x in self.default_ext.split(',')])
                for wordlist in self.get_option('wordlist'):
                        name = os.path.splitext(os.path.basename(wordlist))[0]
                        if self.get_option('tool') == 'feroxbuster':
                                service.add_manual_command('Enumerate files with extensions manually (change the extensions you want to enumerate).', ['feroxbuster -u {http_scheme}://{addressv6}:{port}/ -t ' + self.default_threads + ' -w ' + wordlist + ' -x "' + self.default_ext + '" -v -k -n -q -o "{scandir}/{protocol}_{port}_{http_scheme}_feroxbuster_' + name + '.txt"'])
                        elif self.get_option('tool') == 'gobuster':
                                service.add_manual_command('Enumerate files with extensions manually (change the extensions you want to enumerate).', ['gobuster dir -u {http_scheme}://{addressv6}:{port}/ -t ' + self.default_threads + ' -w ' + wordlist + ' -e -k -x "' + self.default_ext + '" -z -o "{scandir}/{protocol}_{port}_{http_scheme}_gobuster_' + name + '.txt"'])
                        elif self.get_option('tool') == 'dirsearch':
                                if service.target.ipversion == 'IPv6':
                                        error('dirsearch does not support IPv6.')
                                else:
                                        service.add_manual_command('Enumerate files with extensions manually (change the extensions you want to enumerate).', ['dirsearch -u {http_scheme}://{address}:{port}/ -t ' + str(self.default_threads) + ' -e "' + self.default_ext + '" -f -q -w ' + wordlist + ' --format=plain -o "{scandir}/{protocol}_{port}_{http_scheme}_dirsearch_' + name + '.txt"'])
                        elif self.get_option('tool') == 'ffuf':
                                service.add_manual_command('Enumerate files with extensions manually (change the extensions you want to enumerate).', ['ffuf -u {http_scheme}://{addressv6}:{port}/FUZZ -t ' + str(self.default_threads) + ' -w ' + wordlist + ' -e "' + dot_extensions + '" -v -noninteractive | tee {scandir}/{protocol}_{port}_{http_scheme}_ffuf_' + name + '.txt'])
                        elif self.get_option('tool') == 'dirb':
                                service.add_manual_command('Enumerate files with extensions manually (change the extensions you want to enumerate).', ['dirb {http_scheme}://{addressv6}:{port}/ ' + wordlist + ' -l -r -S -X ",' + dot_extensions + '" -o "{scandir}/{protocol}_{port}_{http_scheme}_dirb_' + name + '.txt"'])

class GobusterCommon(ServiceScan):
        def __init__(self):
                super().__init__()
                self.name = "Gobuster using common.txt with file extensions"
                self.slug = 'gobuster-common'
                self.priority = 0
                self.tags = ['default', 'safe', 'long', 'http']

        def configure(self):
                self.default_threads = 10
                self.default_ext = 'txt,html,php,asp,aspx,jsp'

                self.match_service_name('^http')
                self.match_service_name('^nacn_http$', negative_match=True)

        def check(self):
                tool = 'gobuster'
                if which('gobuster') is None:
                        error('The gobuster program could not be found. Make sure it is installed. (On Kali, run: sudo apt install gobuster)')

        async def run(self, service):
                wordlist = "/usr/share/seclists/Discovery/Web-Content/common.txt"
                name = os.path.splitext(os.path.basename(wordlist))[0]
                await service.execute('gobuster dir -u {http_scheme}://{addressv6}:{port}/ -t ' + self.default_threads + ' -w ' + wordlist + ' -e -k -x "' + self.default_ext + '" -z -o "{scandir}/{protocol}_{port}_{http_scheme}_gobuster_' + name + '.txt"')

class CustomWebSubdomainEnumeration(ServiceScan):
        def __init__(self):
                super().__init__()
                self.name = "Custom Web Subdomain Enumerator"
                self.slug = 'custom-web-subdomain-enumerator'
                self.priority = 0
                self.tags = ['default', 'safe', 'long', 'http']

        def configure(self):
                self.match_service_name('^http')
                self.match_service_name('^nacn_http$', negative_match=True)

        def check(self):
                if which('ffuf') is None:
                        error('The ffuf program could not be found. Make sure it is installed. (On Kali, run: sudo apt install ffuf)')

        def manual(self, service, plugin_was_run):
                service.add_manual_command('(ffuf) Enumerate subdomains of a web server (you will probably have to filter out incorrect entries and change the HOST header)', ['ffuf -w /usr/share/seclists/Discovery/DNS/subdomains-top1million-110000.txt -H "HOST:FUZZ.{address}" -u {scheme}://{address}:{port} -s 2>&1 | tee {scandir}/{protocol}_{port}_{scheme}_ffuf_enumerate_subdomain.txt'])     

I've put another class in it in the mean time, but note that this very same problem occurred initially when I had made the first class, DirBusterCustom. So if you want to, you can go to issue #110 and find a smaller test case. I think I posted the source code there.

[Tib3rius]

Ok, I believe I fixed the issue this time in v2.0.3 (committed to the main branch). The exceptions are reported now. I took the liberty of fixing the errors in your code. You'd forgotten to convert the default_threads to a string in a couple of places, and you were using {scheme} instead of {http_scheme} in some of the commands as well. This should work:

from autorecon.plugins import ServiceScan
from autorecon.io import error, info, fformat
from shutil import which
import os

class DirBusterCustom(ServiceScan):

        def __init__(self):
                super().__init__()
                self.name = "Custom Directory Buster"
                self.slug = 'dirbuster-manual-extensions'
                self.priority = 0
                self.tags = ['default', 'safe', 'long', 'http']

        def configure(self):
                self.add_choice_option('tool', default='gobuster', choices=['feroxbuster', 'gobuster', 'dirsearch', 'ffuf', 'dirb'], help='The tool to use for directory busting. Default: %(default)s')
                self.add_list_option('wordlist', default=['/home/x90slide/resources/infosec-knowledge/wordlists/web_content/combined_words.txt'], help='The wordlist(s) to use for the custom HTTP scan plugin. Default: %(default)s')

                self.default_threads = 10
                self.default_ext = 'txt,html,php,asp,aspx,jsp'

                self.match_service_name('^http')
                self.match_service_name('^nacn_http$', negative_match=True)

        def check(self):
                tool = self.get_option('tool')
                if tool == 'feroxbuster':
                        if which('feroxbuster') is None:
                                error('The feroxbuster program could not be found. Make sure it is installed. (On Kali, run: sudo apt install feroxbuster)')
                elif tool == 'gobuster':
                        if which('gobuster') is None:
                                error('The gobuster program could not be found. Make sure it is installed. (On Kali, run: sudo apt install gobuster)')
                elif tool == 'dirsearch':
                        if which('dirsearch') is None:
                                error('The dirsearch program could not be found. Make sure it is installed. (On Kali, run: sudo apt install dirsearch)')

        def manual(self, service, plugin_was_run):
                dot_extensions = ','.join(['.' + x for x in self.default_ext.split(',')])
                for wordlist in self.get_option('wordlist'):
                        name = os.path.splitext(os.path.basename(wordlist))[0]
                        if self.get_option('tool') == 'feroxbuster':
                                service.add_manual_command('Enumerate files with extensions manually (change the extensions you want to enumerate).', ['feroxbuster -u {http_scheme}://{addressv6}:{port}/ -t ' + str(self.default_threads) + ' -w ' + wordlist + ' -x "' + self.default_ext + '" -v -k -n -q -o "{scandir}/{protocol}_{port}_{http_scheme}_feroxbuster_' + name + '.txt"'])
                        elif self.get_option('tool') == 'gobuster':
                                service.add_manual_command('Enumerate files with extensions manually (change the extensions you want to enumerate).', ['gobuster dir -u {http_scheme}://{addressv6}:{port}/ -t ' + str(self.default_threads) + ' -w ' + wordlist + ' -e -k -x "' + self.default_ext + '" -z -o "{scandir}/{protocol}_{port}_{http_scheme}_gobuster_' + name + '.txt"'])
                        elif self.get_option('tool') == 'dirsearch':
                                if service.target.ipversion == 'IPv6':
                                        error('dirsearch does not support IPv6.')
                                else:
                                        service.add_manual_command('Enumerate files with extensions manually (change the extensions you want to enumerate).', ['dirsearch -u {http_scheme}://{address}:{port}/ -t ' + str(self.default_threads) + ' -e "' + self.default_ext + '" -f -q -w ' + wordlist + ' --format=plain -o "{scandir}/{protocol}_{port}_{http_scheme}_dirsearch_' + name + '.txt"'])
                        elif self.get_option('tool') == 'ffuf':
                                service.add_manual_command('Enumerate files with extensions manually (change the extensions you want to enumerate).', ['ffuf -u {http_scheme}://{addressv6}:{port}/FUZZ -t ' + str(self.default_threads) + ' -w ' + wordlist + ' -e "' + dot_extensions + '" -v -noninteractive | tee {scandir}/{protocol}_{port}_{http_scheme}_ffuf_' + name + '.txt'])
                        elif self.get_option('tool') == 'dirb':
                                service.add_manual_command('Enumerate files with extensions manually (change the extensions you want to enumerate).', ['dirb {http_scheme}://{addressv6}:{port}/ ' + wordlist + ' -l -r -S -X ",' + dot_extensions + '" -o "{scandir}/{protocol}_{port}_{http_scheme}_dirb_' + name + '.txt"'])

class GobusterCommon(ServiceScan):
        def __init__(self):
                super().__init__()
                self.name = "Gobuster using common.txt with file extensions"
                self.slug = 'gobuster-common'
                self.priority = 0
                self.tags = ['default', 'safe', 'long', 'http']

        def configure(self):
                self.default_threads = 10
                self.default_ext = 'txt,html,php,asp,aspx,jsp'

                self.match_service_name('^http')
                self.match_service_name('^nacn_http$', negative_match=True)

        def check(self):
                tool = 'gobuster'
                if which('gobuster') is None:
                        error('The gobuster program could not be found. Make sure it is installed. (On Kali, run: sudo apt install gobuster)')

        async def run(self, service):
                wordlist = "/usr/share/seclists/Discovery/Web-Content/common.txt"
                name = os.path.splitext(os.path.basename(wordlist))[0]
                await service.execute('gobuster dir -u {http_scheme}://{addressv6}:{port}/ -t ' + str(self.default_threads) + ' -w ' + wordlist + ' -e -k -x "' + self.default_ext + '" -z -o "{scandir}/{protocol}_{port}_{http_scheme}_gobuster_' + name + '.txt"')

class CustomWebSubdomainEnumeration(ServiceScan):
        def __init__(self):
                super().__init__()
                self.name = "Custom Web Subdomain Enumerator"
                self.slug = 'custom-web-subdomain-enumerator'
                self.priority = 0
                self.tags = ['default', 'safe', 'long', 'http']

        def configure(self):
                self.match_service_name('^http')
                self.match_service_name('^nacn_http$', negative_match=True)

        def check(self):
                if which('ffuf') is None:
                        error('The ffuf program could not be found. Make sure it is installed. (On Kali, run: sudo apt install ffuf)')

        def manual(self, service, plugin_was_run):
                service.add_manual_command('(ffuf) Enumerate subdomains of a web server (you will probably have to filter out incorrect entries and change the HOST header)', ['ffuf -w /usr/share/seclists/Discovery/DNS/subdomains-top1million-110000.txt -H "HOST:FUZZ.{address}" -u {http_scheme}://{address}:{port} -s 2>&1 | tee {scandir}/{protocol}_{port}_{http_scheme}_ffuf_enumerate_subdomain.txt'])

[heinosasshallik]

Ah, thanks! I left the string errors in purposefully, but I didn't know about the http_scheme thing.

I updated to 2.0.3 (and confirmed). Still the same issue, but this time CTRL+C doesn't give any errors. Also double checked that port 80 (among others) are still up on the HTB intelligence machine, and they are.

[Tib3rius]

By the same issue, you mean it's not running the plugins? Can you share your command and your config.toml file (if you're using a custom one)? Sorry if I'm asking for the same stuff again, it's just when I ran it against my local machine (with HTTP 80 open) the plugins ran fine.

[heinosasshallik]

Yeah, I mean that it's not running the plugins (except for the DNS one).

Everything is default and generated by autorecon, except for the purposely faulty plugin. Here's the folder from /root/.config/Autorecon:

autorecon.zip

The command I'm running is sudo autorecon intelligence -v --output dirty | tee dirty.txt

[Tib3rius]

Hi @heinosasshallik,

It looks like http_server_broken.py isn't in the plugins directory in /root/.config/AutoRecon. Can you move it to the plugins directory and try again?

Alternatively you can create a separate plugins directory anywhere on your machine, move the file there, and use: --add-plugins-dir /path/to/new/plugins/dir to specify the other directory.

[heinosasshallik]

Oh that's quite embarrassing. Sorry about that.

Moved it to the correct folder and yes, it's giving exceptions as expected. I suppose it's still strange that it didn't work when the broken plugin was in the parent folder, but that's not that important, I suppose.

Thank you for fixing the issue! :)