Fix encoding issue with non-ASCII characters

cdelafuente-r7 commented 1 year ago

This is related to this Metasploit issue.

This PR addresses an authentication error when usernames with non-ASCII characters are used. This PR fixes two things:

Metasploit converts strings containing non-ASCII characters to ASCII-8BIT encoding.

Calling String#encode on those strings were throwing a Encoding::UndefinedConversionError exception. To address this, a safe_encode wrapper method has been added. This method handles this specific Metasploit case and prevents it from breaking. This wrapper simply catches Encoding exceptions and #force_encode instead of #encode the string. This might not be the best solution, but since it seems to only happen in this specific scenario, I'm confident it is safe to do.

Every calls to String#encode on a string that could come from a user input directly have been replace by #safe_encode.

Fix NTLMv2 hash when username contains non-ASCII characters in the rubyntlm library

See https://github.com/WinRb/rubyntlm/issues/55 for details. A PR has been submitted, but since we want this to be fixed ASAP, the fix has been added to RubySMB directly. This PR adds the patched rubyntlm's ntlmv2_hash method and forces the RubySMB::Client to use RubySMB::NTLM instead of rubyntlm library.

Before the fix

❯ ruby examples/tree_connect.rb --username юзер --password 123456 10.0.0.68 IPC$
SMB3 : (0xc000006d) STATUS_LOGON_FAILURE: The attempted logon is invalid. This is either due to a bad username or authentication information.
Authentication failed!

After the fix

❯ ruby examples/tree_connect.rb --username юзер --password 123456 10.0.0.168 IPC$
SMB3 : (0x00000000) STATUS_SUCCESS: The operation completed successfully.
Connected to \\192.168.144.168\IPC$ successfully!

cgranleese-r7 commented 1 year ago

Hi @cdelafuente-r7 , I was working on some ASCII/UTF-8 issues and hooked up your changes to see if they would fix an issue we were running into on a call. I'll add a before and after comparing upstream-master and your branch to show the differences I was getting.

Before

Encoding exception is raised:

msf6 exploit(windows/smb/psexec) > run smb://H��l��ne:password@192.168.175.131

[*] Started reverse TCP handler on 192.168.8.125:4444
[*] 192.168.175.131:445 - Connecting to the server...
[*] 192.168.175.131:445 - Authenticating to 192.168.175.131:445 as user 'Hélène'...
[-] 192.168.175.131:445 - Exploit failed [no-access]: Rex::Proto::SMB::Exceptions::LoginError Login Failed: "\xC3" from ASCII-8BIT to UTF-8
[*] Exploit completed, but no session was created.

Note: the username H��l��ne was originally pasted as Hélène originally, that seems like an unrelated issue*

After

The exception is now different:

msf6 exploit(windows/smb/psexec) > run smb://H��l��ne:password@192.168.175.131

[*] Started reverse TCP handler on 192.168.8.125:4444
[*] 192.168.175.131:445 - Connecting to the server...
[*] 192.168.175.131:445 - Authenticating to 192.168.175.131:445 as user 'Hélène'...
[-] 192.168.175.131:445 - Exploit failed [no-access]: Rex::Proto::SMB::Exceptions::LoginError Login Failed: incompatible character encodings: ASCII-8BIT and UTF-8
[*] Exploit completed, but no session was created.

Note: the username H��l��ne was originally pasted as Hélène originally, that seems like an unrelated issue*

Stacktrace

Original stacktrace from ruby_smb:

[3] pry(#<Rex::Proto::SMB::SimpleClient>)> puts e.backtrace
/Users/cgranleese/.rvm/gems/ruby-3.0.2@metasploit-framework/bundler/gems/ruby_smb-4c4531757f59/lib/ruby_smb/ntlm/client.rb:13:in `join'
/Users/cgranleese/.rvm/gems/ruby-3.0.2@metasploit-framework/bundler/gems/ruby_smb-4c4531757f59/lib/ruby_smb/ntlm/client.rb:13:in `serialize'
/Users/cgranleese/.rvm/gems/ruby-3.0.2@metasploit-framework/bundler/gems/ruby_smb-4c4531757f59/lib/ruby_smb/client/authentication.rb:359:in `smb2_ntlmssp_auth_packet'
/Users/cgranleese/.rvm/gems/ruby-3.0.2@metasploit-framework/bundler/gems/ruby_smb-4c4531757f59/lib/ruby_smb/client/authentication.rb:342:in `smb2_ntlmssp_authenticate'
/Users/cgranleese/.rvm/gems/ruby-3.0.2@metasploit-framework/bundler/gems/ruby_smb-4c4531757f59/lib/ruby_smb/client/authentication.rb:222:in `smb2_authenticate'
/Users/cgranleese/.rvm/gems/ruby-3.0.2@metasploit-framework/bundler/gems/ruby_smb-4c4531757f59/lib/ruby_smb/client/authentication.rb:22:in `authenticate'
/Users/cgranleese/.rvm/gems/ruby-3.0.2@metasploit-framework/bundler/gems/ruby_smb-4c4531757f59/lib/ruby_smb/client.rb:432:in `session_setup'
/Users/cgranleese/Documents/code/metasploit-framework/lib/rex/proto/smb/simple_client.rb:100:in `login'
/Users/cgranleese/Documents/code/metasploit-framework/lib/msf/core/exploit/remote/smb/client.rb:152:in `smb_login'
/Users/cgranleese/Documents/code/metasploit-framework/modules/exploits/windows/smb/psexec.rb:132:in `exploit'
/Users/cgranleese/Documents/code/metasploit-framework/lib/msf/core/exploit_driver.rb:228:in `job_run_proc'
/Users/cgranleese/Documents/code/metasploit-framework/lib/msf/core/exploit_driver.rb:181:in `run'
/Users/cgranleese/Documents/code/metasploit-framework/lib/msf/base/simple/exploit.rb:144:in `exploit_simple'
/Users/cgranleese/Documents/code/metasploit-framework/lib/msf/base/simple/exploit.rb:171:in `exploit_simple'
/Users/cgranleese/Documents/code/metasploit-framework/lib/msf/ui/console/command_dispatcher/exploit.rb:45:in `exploit_single'
/Users/cgranleese/Documents/code/metasploit-framework/lib/msf/ui/console/command_dispatcher/exploit.rb:182:in `cmd_exploit'
/Users/cgranleese/Documents/code/metasploit-framework/lib/rex/ui/text/dispatcher_shell.rb:581:in `run_command'
/Users/cgranleese/Documents/code/metasploit-framework/lib/rex/ui/text/dispatcher_shell.rb:530:in `block in run_single'
/Users/cgranleese/Documents/code/metasploit-framework/lib/rex/ui/text/dispatcher_shell.rb:524:in `each'
/Users/cgranleese/Documents/code/metasploit-framework/lib/rex/ui/text/dispatcher_shell.rb:524:in `run_single'
/Users/cgranleese/Documents/code/metasploit-framework/lib/rex/ui/text/shell.rb:162:in `run'
/Users/cgranleese/Documents/code/metasploit-framework/lib/metasploit/framework/command/console.rb:48:in `start'
/Users/cgranleese/Documents/code/metasploit-framework/lib/metasploit/framework/command/base.rb:82:in `start'
./msfconsole:23:in `<main>'

and this is from framework logs:

[10/07/2022 10:59:09] [d(0)] core: SMB version(s) to negotiate: [1, 2, 3]
[10/07/2022 10:59:09] [d(0)] core: Negotiated SMB version: SMB3
[10/07/2022 10:59:09] [e(0)] core: Exploit failed (windows/smb/psexec): Rex::Proto::SMB::Exceptions::LoginError Login Failed: incompatible character encodings: ASCII-8BIT and UTF-8 - Rex::Proto::SMB::Exceptions::LoginError Login Failed: incompatible character encodings: ASCII-8BIT and UTF-8
Call stack:
/Users/cgranleese/Documents/code/metasploit-framework/lib/rex/proto/smb/simple_client.rb:110:in `rescue in login'
/Users/cgranleese/Documents/code/metasploit-framework/lib/rex/proto/smb/simple_client.rb:62:in `login'
/Users/cgranleese/Documents/code/metasploit-framework/lib/msf/core/exploit/remote/smb/client.rb:152:in `smb_login'
/Users/cgranleese/Documents/code/metasploit-framework/modules/exploits/windows/smb/psexec.rb:132:in `exploit'
/Users/cgranleese/Documents/code/metasploit-framework/lib/msf/core/exploit_driver.rb:228:in `job_run_proc'
/Users/cgranleese/Documents/code/metasploit-framework/lib/msf/core/exploit_driver.rb:181:in `run'
/Users/cgranleese/Documents/code/metasploit-framework/lib/msf/base/simple/exploit.rb:144:in `exploit_simple'
/Users/cgranleese/Documents/code/metasploit-framework/lib/msf/base/simple/exploit.rb:171:in `exploit_simple'
/Users/cgranleese/Documents/code/metasploit-framework/lib/msf/ui/console/command_dispatcher/exploit.rb:45:in `exploit_single'
/Users/cgranleese/Documents/code/metasploit-framework/lib/msf/ui/console/command_dispatcher/exploit.rb:182:in `cmd_exploit'
/Users/cgranleese/Documents/code/metasploit-framework/lib/rex/ui/text/dispatcher_shell.rb:581:in `run_command'
/Users/cgranleese/Documents/code/metasploit-framework/lib/rex/ui/text/dispatcher_shell.rb:530:in `block in run_single'
/Users/cgranleese/Documents/code/metasploit-framework/lib/rex/ui/text/dispatcher_shell.rb:524:in `each'
/Users/cgranleese/Documents/code/metasploit-framework/lib/rex/ui/text/dispatcher_shell.rb:524:in `run_single'
/Users/cgranleese/Documents/code/metasploit-framework/lib/rex/ui/text/shell.rb:162:in `run'
/Users/cgranleese/Documents/code/metasploit-framework/lib/metasploit/framework/command/console.rb:48:in `start'
/Users/cgranleese/Documents/code/metasploit-framework/lib/metasploit/framework/command/base.rb:82:in `start'
./msfconsole:23:in `<main>'

Was this PR in isolation able to fix the original metasploit-framework issue, or are there additional changes required in metasploit?

cdelafuente-r7 commented 1 year ago

Thanks @cgranleese-r7 for reporting this issue. I tracked down to a rubyntlm library encoding issue. I monkey-patched it and it should be okay now, but I'm not sure if it is the best to fix it. That said, please, would mind retesting with this in place and let me know if it fixes the issue for you too?

cgranleese-r7 commented 1 year ago

Re-tested your branch and it seems your latest changes fixed the issue on my environment as well.

Thanks for looking into this 👍

smcintyre-r7 commented 1 year ago

I was able to reproduce the original issue and validate that the proposed changes fix it.

Fixed / After Patch

``` msf6 exploit(windows/smb/psexec) > run [*] Started reverse TCP handler on 192.168.159.128:4444 [*] 192.168.159.10:445 - Connecting to the server... [*] 192.168.159.10:445 - Authenticating to 192.168.159.10:445 as user 'järvalv'... [*] 192.168.159.10:445 - Selecting PowerShell target [*] 192.168.159.10:445 - Executing the payload... [+] 192.168.159.10:445 - Service start timed out, OK if running a command or non-service executable... [*] Sending stage (200774 bytes) to 192.168.159.10 [*] Meterpreter session 1 opened (192.168.159.128:4444 -> 192.168.159.10:49773) at 2022-11-17 09:09:12 -0500 meterpreter > ```

Broken / Before Patch

``` msf6 exploit(windows/smb/psexec) > run [*] Started reverse TCP handler on 192.168.159.128:4444 [*] 192.168.159.10:445 - Connecting to the server... [*] 192.168.159.10:445 - Authenticating to 192.168.159.10:445 as user 'järvalv'... [-] 192.168.159.10:445 - Exploit failed [no-access]: Rex::Proto::SMB::Exceptions::LoginError Login Failed: "\xC3" from ASCII-8BIT to UTF-8 [*] Exploit completed, but no session was created. msf6 exploit(windows/smb/psexec) > ```

gwillcox-r7 commented 1 year ago

I see we still have some outstanding comments here. Should we open another PR to address those? Jeffery's points seem valid here.

jmartin-tech commented 1 year ago

Yes that can be a follow up PR, this at least gets the issue addressed as landed.

rapid7 / ruby_smb