Exception when executing a lot of SMB targets: IndexError: list index out of range

joaovarelas commented 1 week ago

Describe the bug The output contains a couple of exceptions and stacktraces but seem to work for some hosts.

To Reproduce Steps to reproduce the behavior i.e.: Command: netexec smb lots-of-hosts-list.txt -u username -p password Resulted in:

[16:32:13] ERROR    Exception while calling proto_flow() on target 10.1.21.11: list index out of range                                                                           connection.py:174
                    ╭─────────────────────────────────────────────────────────── Traceback (most recent call last) ────────────────────────────────────────────────────────────╮                  
                    │ /home/user/.local/share/pipx/venvs/netexec/lib/python3.12/site-packages/nxc/connection.py:166 in __init__                                                  │                  
                    │                                                                                                                                                          │                  
                    │   163 │   │   self.logger.info(f"Socket info: host={self.host}, hostname={self.hostname},                                                                │                  
                    │       kerberos={self.kerberos}, ipv6={self.is_ipv6}, link-local                                                                                          │                  
                    │       ipv6={self.is_link_local_ipv6}")                                                                                                                   │                  
                    │   164 │   │                                                                                                                                              │                  
                    │   165 │   │   try:                                                                                                                                       │                  
                    │ ❱ 166 │   │   │   self.proto_flow()                                                                                                                      │                  
                    │   167 │   │   except Exception as e:                                                                                                                     │                  
                    │   168 │   │   │   if "ERROR_DEPENDENT_SERVICES_RUNNING" in str(e):                                                                                       │                  
                    │   169 │   │   │   │   self.logger.error(f"Exception while calling proto_flow() on target                                                                 │                  
                    │       {target}: {e}")                                                                                                                                    │                  
                    │                                                                                                                                                          │                  
                    │ /home/user/.local/share/pipx/venvs/netexec/lib/python3.12/site-packages/nxc/connection.py:232 in proto_flow                                                │                  
                    │                                                                                                                                                          │                  
                    │   229 │   │   else:                                                                                                                                      │                  
                    │   230 │   │   │   self.logger.debug("Created connection object")                                                                                         │                  
                    │   231 │   │   │   self.enum_host_info()                                                                                                                  │                  
                    │ ❱ 232 │   │   │   if self.print_host_info() and (self.login() or (self.username == "" and                                                                │                  
                    │       self.password == "")):                                                                                                                             │                  
                    │   233 │   │   │   │   if hasattr(self.args, "module") and self.args.module:                                                                              │                  
                    │   234 │   │   │   │   │   self.load_modules()                                                                                                            │                  
                    │   235 │   │   │   │   │   self.logger.debug("Calling modules")                                                                                           │                  
                    │                                                                                                                                                          │                  
                    │ /home/user/.local/share/pipx/venvs/netexec/lib/python3.12/site-packages/nxc/connection.py:560 in login                                                     │                  
                    │                                                                                                                                                          │                  
                    │   557 │   │   if not self.args.no_bruteforce:                                                                                                            │                  
                    │   558 │   │   │   for secr_index, secr in enumerate(secret):                                                                                             │                  
                    │   559 │   │   │   │   for user_index, user in enumerate(username):                                                                                       │                  
                    │ ❱ 560 │   │   │   │   │   if self.try_credentials(domain[user_index], user, owned[user_index],                                                           │                  
                    │       secr, cred_type[secr_index], data[secr_index]):                                                                                                    │                  
                    │   561 │   │   │   │   │   │   owned[user_index] = True                                                                                                   │                  
                    │   562 │   │   │   │   │   │   if not self.args.continue_on_success:                                                                                      │                  
                    │   563 │   │   │   │   │   │   │   return True                                                                                                            │                  
                    │                                                                                                                                                          │                  
                    │ /home/user/.local/share/pipx/venvs/netexec/lib/python3.12/site-packages/nxc/connection.py:495 in try_credentials                                           │                  
                    │                                                                                                                                                          │                  
                    │   492 │   │   │   │   │   return self.kerberos_login(domain, username, secret, "", "",                                                                   │                  
                    │       self.kdcHost, False)                                                                                                                               │                  
                    │   493 │   │   │   │   elif hasattr(self.args, "domain"):  # Some protocols don't use domain                                                              │                  
                    │       for login                                                                                                                                          │                  
                    │   494 │   │   │   │   │   self.logger.debug("Trying to authenticate using plaintext with                                                                 │                  
                    │       domain")                                                                                                                                           │                  
                    │ ❱ 495 │   │   │   │   │   return self.plaintext_login(domain, username, secret)                                                                          │                  
                    │   496 │   │   │   │   elif self.args.protocol == "ssh":                                                                                                  │                  
                    │   497 │   │   │   │   │   self.logger.debug("Trying to authenticate using plaintext over SSH")                                                           │                  
                    │   498 │   │   │   │   │   return self.plaintext_login(username, secret, data)                                                                            │                  
                    │                                                                                                                                                          │                  
                    │ /home/user/.local/share/pipx/venvs/netexec/lib/python3.12/site-packages/nxc/protocols/smb.py:423 in plaintext_login                                        │                  
                    │                                                                                                                                                          │                  
                    │    420 │   │   │   self.logger.debug(f"Adding credential:                                                                                                │                  
                    │        {domain}/{self.username}:{self.password}")                                                                                                        │                  
                    │    421 │   │   │   self.db.add_credential("plaintext", domain, self.username, self.password)                                                             │                  
                    │    422 │   │   │   user_id = self.db.get_credential("plaintext", domain, self.username,                                                                  │                  
                    │        self.password)                                                                                                                                    │                  
                    │ ❱  423 │   │   │   host_id = self.db.get_hosts(self.host)[0].id                                                                                          │                  
                    │    424 │   │   │   self.db.add_loggedin_relation(user_id, host_id)                                                                                       │                  
                    │    425 │   │   │                                                                                                                                         │                  
                    │    426 │   │   │   out = f"{domain}\\{self.username}:{process_secret(self.password)}                                                                     │                  
                    │        {self.mark_guest()}{self.mark_pwned()}"                                                                                                           │                  
                    ╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯                  
                    IndexError: list index out of range

Expected behavior No exceptions

Screenshots

NetExec info

OS: Kali
Version of nxc: v1.3.0 Need for Speed
Installed from: pipx

termanix commented 1 week ago

How many hosts you are talking about? Cause while I testing to banks, I didn't get error like this.

joaovarelas commented 1 week ago

How many hosts you are talking about? Cause while I testing to banks, I didn't get error like this.

Im doing scan to multiple /24 networks, but I would say around 250 to 300 hosts at least with port 445 reachable. I'm not sure what triggers this error but for less hosts it does not happen...

Additionally if I scan the single host that provoked the exeception, the output is fine.

EDIT: I noticed i did not specify domain -d domain but doesn't nxc assume the machine domain by default?

termanix commented 1 week ago

If the txt file contains subnets, this could be the reason. I always find active hosts first and then start testing. I'm talking about thousands hosts btw.

joaovarelas commented 1 week ago

If the txt file contains subnets, this could be the reason. I always find active hosts first and then start testing. I'm talking about thousands hosts btw.

That make sense. I read the exception with attention and the issue shall be here, by retreiving non existent index [0]

❱  423 │   │   │   host_id = self.db.get_hosts(self.host)[0].id

Will have a look at this.

joaovarelas commented 1 week ago

So today I am trying several things:

nxc smb hosts.txt -u user -p password where hosts.txt contains lots of /24 CIDR addresses
nxc smb hosts.txt -u user -p password where hosts.txt contains lots of ipv4 addresses without network prefix
same as above but with couple of empty lines, whitespaces, bad-formatted ipv4 addresses

I am not getting that exception now... Weird

termanix commented 1 week ago

Working with subnets could lead to such situations. Subnet scanning is much larger than host scanning. I'm glad you didn't encounter it. However, the maitainers will of course do in-depth research when they are available. I will examine it too in detail when available.

NeffIsBack commented 1 week ago

Can you give me your version info?

NeffIsBack commented 1 week ago

The issue is that apparently the host that you are trying to log into does not exist in the database. Though this should not happen as it gets added in enum_host_info earlier.

My best guess would be that you ran into a race condition with SQLite where you tried to read from the db before the host was added. Perhaps because SQLite was overwhelmed by all these connections?

Btw if you want to retest the issue you have to clear the database (or remove the nxc/workspace folder entirely). Otherwise the host was added by your previous scan

NeffIsBack commented 1 week ago

My best guess would be that you ran into a race condition with SQLite where you tried to read from the db before the host was added. Perhaps because SQLite was overwhelmed by all these connections?

Though, concurrency issues in the database should have been fixed with #400

joaovarelas commented 1 week ago

The issue is that apparently the host that you are trying to log into does not exist in the database. Though this should not happen as it gets added in enum_host_info earlier.

My best guess would be that you ran into a race condition with SQLite where you tried to read from the db before the host was added. Perhaps because SQLite was overwhelmed by all these connections?

Btw if you want to retest the issue you have to clear the database (or remove the nxc/workspace folder entirely). Otherwise the host was added by your previous scan

Thanks @NeffIsBack at first I had the impression it was indeed some issue with concurrency. I will clear the database and try to reproduce again.

EDIT: 1 out of 5 runs i get the exception with the same exact command, after clearing and initiating the empty database.

Kali 2024.1
Python 3.12.6
1.3.0 - NeedForSpeed - 6d4fdfd

Command: nxc smb hosts.txt -u validuser -p validpass. hosts.txt has 160 ipv4 address, i believe all of them with port 445 reachable.

Im now trying to trigger the exception with --debug but without success.

joaovarelas commented 1 week ago

Ok so triggered the exception with --debug but don't really know if it will be helpful. I had to obfuscate some strings on the log, nxc-exception.log

NeffIsBack commented 1 week ago

Took a look at the log but it is really hard to debug without being able to test it. The question would be why this host wasn't in the database in the first place. If you still have access you could:

Delete the DB
Trigger the error
Take a look at the DB to see if the host was added at some point
Take a look at the debug log to see if the protocol maybe failed to add the host to the db

Pennyw0rth / NetExec

Exception when executing a lot of SMB targets: IndexError: list index out of range #454