search

fortra / impacket

Impacket is a collection of Python classes for working with network protocols.
https://www.coresecurity.com
Other
13.37k stars 3.56k forks source link

utf-8 issues with examples #154

Closed maaaaz closed 5 years ago

maaaaz commented 8 years ago

Hello @asolino, I'll use this thread to report various utf-8 issues :)

Let's start with smbclient.py:

$ smbclient.py <login>:<pwd>@192.168.11.144
# shares
ADMIN$
C$
IPC$
NETLOGON
share
SYSVOL
àéyoloshare
# cd àéyoloshare
[-] 'utf8' codec can't decode byte 0x85 in position 3: invalid start byte

I know utf handling is a pain...That's a long struggle with CrackMapExec (hello @byt3bl33d3r) :)

Cheers !

asolino commented 8 years ago

@maaaaz thanks for reporting it!

Yeah.. utf and Python is a nigthmare, but we want to support it, so keep them coming..

cheers mate.

asolino commented 8 years ago

hey @maaaaz

I can't reproduce it with the master version (tried it both from Kali and OSX). What OS are you using? Can you please test this using master branch?

thanks

maaaaz commented 8 years ago

Ah sorry I totally forgot to say that the share àéyoloshare was on a Windows Server 2012.

asolino commented 8 years ago

But where are you running smbclient.py from?

maaaaz commented 8 years ago

From the windows-compiled version of smbclient, on a french-localed Windows 10. I'll check from a Linux environment but I expect to face the same issue.

asolino commented 8 years ago

@maaaaz that'd be great mate. As I said I can't reproduce it both on Kali and OSX.

merci!

maaaaz commented 8 years ago

You were right, results are different.

root@kali:~# smbclient.py '<login>:<pass>@192.168.11.144'
Impacket v0.9.15-dev - Copyright 2002-2016 Core Security Technologies

Type help for list of commands
# shares
ADMIN$
C$
IPC$
NETLOGON
share
SYSVOL
àéyoloshare
# use àéyoloshare
# ls
drw-rw-rw-          0  Sun Apr  3 11:58:17 2016 .
drw-rw-rw-          0  Sun Apr  3 11:58:17 2016 ..
(venvimpacketwin) Z:\Partage\impacket_fork\examples>python smbclient.py <login>:<pass>@192.168.11.144
Impacket v0.9.15-dev - Copyright 2002-2016 Core Security Technologies

Type help for list of commands
#
# shares
ADMIN$
C$
IPC$
NETLOGON
share
SYSVOL
àéyoloshare
# use àéyoloshare
[-] 'utf8' codec can't decode byte 0x85 in position 4: invalid start byte
maaaaz commented 8 years ago

Then for smbclient, more bugs with the Windows version:

# cd encoding2
# ls
drw-rw-rw-          0  Sun Nov 29 15:11:08 2015 .
drw-rw-rw-          0  Sun Nov 29 15:11:08 2015 ..
-rw-rw-rw-         25  Sun Nov 29 15:11:08 2015 credz.txt
-rw-rw-rw-          0  Sun Nov 29 15:11:08 2015 àlol.txt
-rw-rw-rw-          0  Sun Nov 29 15:11:08 2015 àâæçéèêëïî.txt
drw-rw-rw-          0  Sun Nov 29 15:11:08 2015 àâæçéèêëïîô
[-] 'charmap' codec can't encode characters in position 59-61: character maps to <undefined>

While the Linux version is ok:

# cd encoding2
# ls
drw-rw-rw-          0  Sun Nov 29 14:11:08 2015 .
drw-rw-rw-          0  Sun Nov 29 14:11:08 2015 ..
-rw-rw-rw-         25  Sun Nov 29 14:11:08 2015 credz.txt
-rw-rw-rw-          0  Sun Nov 29 14:11:08 2015 àlol.txt
-rw-rw-rw-          0  Sun Nov 29 14:11:08 2015 àâæçéèêëïî.txt
drw-rw-rw-          0  Sun Nov 29 14:11:08 2015 àâæçéèêëïîô
-rw-rw-rw-         33  Sun Nov 29 14:11:08 2015 àâæçéèêëïîôœ—– »« ”’€ÿüûù.txt
maaaaz commented 8 years ago

Then for psexec I'm trying to pass a command with utf-8 chars inside, such as listing information about the àlolé user (also used as connecting user):

(venvcrackmapexec)root@kali:~/Partage/impacket_fork/examples# python psexec.py 'àlolé:<pass>@192.168.11.144' 'cmd.exe /c echo àlolé && net user àlolé /domain' -debug
Impacket v0.9.15-dev - Copyright 2002-2016 Core Security Technologies

[*] Trying protocol 445/SMB...

[*] Requesting shares on 192.168.11.144.....
[-] share 'ADMIN$' is not writable.
[*] Found writable share C$
[*] Uploading file sSzWFEUz.exe
[*] Opening SVCManager on 192.168.11.144.....
[*] Creating service NJZP on 192.168.11.144.....
[*] Starting service NJZP.....
[!] Press help for extra shell commands
A lolAc
The user name could not be found.

More help is available by typing NET HELPMSG 2221.

[*] Process cmd.exe /c echo àlolé && net user àlolé /domain finished with ErrorCode: 0, ReturnCode: 2
[*] Opening SVCManager on 192.168.11.144.....
[*] Stoping service NJZP.....
[*] Removing service NJZP.....
[*] Removing file sSzWFEUz.exe.....
(venvimpacketwin) Z:\Partage\impacket_fork\examples>python psexec.py àlolé:<pass>@192.168.11.144 'cmd.exe /c echo àlolé && net user àlolé' -debug
Impacket v0.9.15-dev - Copyright 2002-2016 Core Security Technologies

[*] Trying protocol 445/SMB...

[*] Requesting shares on 192.168.11.144.....
[-] share 'ADMIN$' is not writable.
[*] Found writable share C$
[*] Uploading file jfDwXxEU.exe
[*] Opening SVCManager on 192.168.11.144.....
[*] Creating service WVFu on 192.168.11.144.....
[*] Starting service WVFu.....
[!] Press help for extra shell commands
[*] Process 'cmd.exe /c echo ÓlolÚ finished with ErrorCode: 1, ReturnCode: 0
[*] Opening SVCManager on 192.168.11.144.....
[*] Stoping service WVFu.....
[*] Removing service WVFu.....
[*] Removing file jfDwXxEU.exe.....

This observation about passing utf-8 parameters might be valid for other examples.

Also keep in mind that inputing utf-8 commands in a psexec session works flawlessly (Linux and Windows):

(venvimpacketwin) Z:\Partage\impacket_fork\examples>python psexec.py àlolé:<pass>@192.168.11.144
Impacket v0.9.15-dev - Copyright 2002-2016 Core Security Technologies

[*] Trying protocol 445/SMB...

[*] Requesting shares on 192.168.11.144.....
[-] share 'ADMIN$' is not writable.
[*] Found writable share C$
[*] Uploading file CiZeKNvL.exe
[*] Opening SVCManager on 192.168.11.144.....
[*] Creating service paKC on 192.168.11.144.....
[*] Starting service paKC.....
[!] Press help for extra shell commands
Microsoft Windows [Version 6.3.9600]
(c) 2013 Microsoft Corporation. All rights reserved.

C:\Windows\system32>net user àlolé
User name                    àlolé
Full Name                    àlolé
Comment
User's comment
...
maaaaz commented 8 years ago

For secretsdump, there's no blocking bug but rather printing defects :):

python secretsdump.py 'àlolé:<pass>@192.168.11.144' -just-dc-user àlolé -debug
Impacket v0.9.15-dev - Copyright 2002-2016 Core Security Technologies

[*] Dumping Domain Credentials (domain\uid:rid:lmhash:nthash)
...
[+] Calling DRSCrackNames for àlolé 
[+] Calling DRSGetNCChanges for CN=àlolé,CN=Users,DC=ADYOLO,DC=SWAG 
[+] Entering NTDSHashes.__decryptHash
[+] Decrypting hash for user: CN=�lol�,CN=Users,DC=ADYOLO,DC=SWAG
ADYOLO.SWAG\àlolé:1112:<hash>
[+] Leaving NTDSHashes.__decryptHash
(venvimpacketwin) Z:\Partage\impacket_fork\examples>python secretsdump.py àlolé:<pass>@192.168.11.144 -just-dc-use
àlolé -debug
Impacket v0.9.15-dev - Copyright 2002-2016 Core Security Technologies

[*] Dumping Domain Credentials (domain\uid:rid:lmhash:nthash)
...
[+] Calling DRSCrackNames for ÓlolÚ
[+] Calling DRSGetNCChanges for CN=àlolé,CN=Users,DC=ADYOLO,DC=SWAG
[+] Entering NTDSHashes.__decryptHash
[+] Decrypting hash for user: CN=ÓlolÚ,CN=Users,DC=ADYOLO,DC=SWAG
ADYOLO.SWAG\àlolé:1112:<hash>
[+] Leaving NTDSHashes.__decryptHash

It has to be noted that there's not any printing defect with -outputfile :)

maaaaz commented 8 years ago

For wmiexec it seems that the share option does not like utf-8 (my list of share is #156) (observation valid for Linux and Windows) and on Linux there's a defect while printing:

$ python wmiexec.py 'àlolé:<pass>@192.168.11.144' whoami -share share -debug
Impacket v0.9.15-dev - Copyright 2002-2016 Core Security Technologies

[*] SMBv3.0 dialect used
[+] StringBinding: \\\\DC01[\\PIPE\\atsvc]
[+] StringBinding: DC01[49155]
[+] StringBinding: 192.168.11.144[49155]
[+] StringBinding chosen: ncacn_ip_tcp:192.168.11.144[49155]
adyolo\�lol�
$ python wmiexec.py 'àlolé:<pass>@192.168.11.144' whoami -share àéyoloshare -debug
Impacket v0.9.15-dev - Copyright 2002-2016 Core Security Technologies

[*] SMBv3.0 dialect used
[+] StringBinding: \\\\DC01[\\PIPE\\atsvc]
[+] StringBinding: DC01[49155]
[+] StringBinding: 192.168.11.144[49155]
[+] StringBinding chosen: ncacn_ip_tcp:192.168.11.144[49155]
...
maaaaz commented 8 years ago

For wmipersist, the name option also does not like utf-8, while the same option works flawlessly for services:

$ python wmipersist.py 'àlolé:<pass>@192.168.11.144' remove -name àlolé
Impacket v0.9.15-dev - Copyright 2002-2016 Core Security Technologies

[-] 'ascii' codec can't decode byte 0xc3 in position 0: ordinal not in range(128)

Observation valid for Linux and Windows.

maaaaz commented 8 years ago

For wmiquery, the prompt also does not like utf-8:

$ python wmiquery.py 'àlolé:<pass>@192.168.11.144'
Impacket v0.9.15-dev - Copyright 2002-2016 Core Security Technologies

[!] Press help for extra shell commands
WQL> help

     lcd {path}                 - changes the current local directory to {path}
     exit                       - terminates the server process (and this session)
     describe {class}           - describes class
     ! {cmd}                    - executes a local shell cmd

WQL> describe toto
[-] WMI Session Error: code: 0x80041002 - WBEM_E_NOT_FOUND
WQL> describe àlolé
[-] 'ascii' codec can't decode byte 0xc3 in position 0: ordinal not in range(128)

Observation valid for Linux and Windows.

End of the utf-8 journey for now, hope you liked it !

asolino commented 8 years ago

@maaaaz you rock mate! We'll fix them all :P.. be patient tho ;)

maaaaz commented 8 years ago

Héhéhé, glad to hear that :)

A last utf-8 issue, with smbexec (the àéyoloshare share exists):

(venvcrackmapexec)root@kali:~/Partage/impacket_fork/examples# python smbexec.py '<login>:<pass>@192.168.11.144' -debug -share àéyoloshare
Impacket v0.9.15-dev - Copyright 2002-2016 Core Security Technologies

[*] Trying protocol 445/SMB...
[*] Creating service BTOBTO...
[+] Executing %COMSPEC% /Q /c echo cd  ^> \\127.0.0.1\àéyoloshare\__output 2^>^&1 > %TEMP%\execute.bat & %COMSPEC% /Q /c %TEMP%\execute.bat & del %TEMP%\execute.bat
[-] SMB SessionError: STATUS_BAD_NETWORK_NAME({Network Name Not Found} The specified share name cannot be found on the remote server.)
Hitechcomputergeek commented 8 years ago

I'm trying to dump hashes from an ntds.dit file extracted from a Windows Server 2003 domain controller. I'm running this from Ubuntu 16.04:

$ secretsdump.py -ntds ntds.dit -system system.save LOCAL
Impacket v0.9.15-dev - Copyright 2002-2016 Core Security Technologies

[*] Target system bootKey: 0x5cb1281562e07ba215831a48c435ed975
[*] Dumping Domain Credentials (domain\uid:rid:lmhash:nthash)
[*] Searching for pekList, be patient
[-] 'utf16' codec can't decode bytes in position 0-1: illegal UTF-16 surrogate
[*] Cleaning up...
$

This has worked for other domain controllers on Server 2012 before.

psst... python3...

EDIT because it's not worth making a new comment for this: it appears to be working in the latest version.

asolino commented 8 years ago

@Hitechcomputergeek

I see that more being a problem reverse engineering the ESE database rather than a specific encoding error. Did you try running it live against the target using DRSUAPI (default method)?

hehee message taken ;)

maaaaz commented 8 years ago

@asolino you might be right as I think I performed that test with secretsdump and a db having a utf-8 user and it worked flawlessly.

ThePirateWhoSmellsOfSunflowers commented 7 years ago

Hi asolino !

Same problem here with eastern european names and secretsdump.py (v0.9.16-dev) Partial output with -debug

[...]
[+] Calling DRSCrackNames for S-X-X-XX-XXXXXXXXX-XXXXXXXXX-XXXXXXXXXX-XXXXXX 
[+] Calling DRSGetNCChanges for CN=YYYYYY Mirosław,OU=XX,OU=XX,OU=XX,OU=DOMAIN,DC=world,DC=domain,DC=local 
[+] Entering NTDSHashes.__decryptHash
[-] Error while processing user!
[-] chr() arg not in range(256)
--
[+] Calling DRSCrackNames for S-X-X-XX-XXXXXXXXX-XXXXXXXXX-XXXXXXXXXX-XXXXXX 
[+] Calling DRSGetNCChanges for CN=YYYYYY Michał,OU=XX,OU=XX,OU=XX,OU=DOMAIN,DC=world,DC=domain,DC=local 
[+] Entering NTDSHashes.__decryptHash
[-] Error while processing user!
[-] chr() arg not in range(256)
--
[+] Calling DRSCrackNames for S-X-X-XX-XXXXXXXXX-XXXXXXXXX-XXXXXXXXXX-XXXXXX 
[+] Calling DRSGetNCChanges for CN=YYYYYY Rafał,OU=XX,OU=XXX,OU=XX,OU=DOMAIN,DC=world,DC=domain,DC=local 
[+] Entering NTDSHashes.__decryptHash
[-] Error while processing user!
[-] chr() arg not in range(256)
[...]

Thanks Cheers :skull_and_crossbones:

asolino commented 7 years ago

Hey @ThePirateWhoSmellsOfSunflowers thanks for the data!

Can you please add traceback messages and run it again? I want to see what line the exception is being thrown.

You should uncomment the two lines here and here.

Are you executing secretsdump.py from Windows or Unix? (what OS?). Any data about the target OS would be great as well so I can reproduce the issue down here.

ThePirateWhoSmellsOfSunflowers commented 7 years ago

Sorry, Tested on Debian 8.6 against Windows Server 2008 R2 Standard SP1 (6.1 7601). Output with traceback : 

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/impacket/examples/secretsdump.py", line 2051, in dump
    hashesOutputFile)
  File "/usr/local/lib/python2.7/dist-packages/impacket/examples/secretsdump.py", line 1735, in __decryptHash
    LOG.debug('Decrypting hash for user: %s' % record['pmsgOut'][replyVersion]['pNC']['StringName'][:-1])
  File "/usr/local/lib/python2.7/dist-packages/impacket/dcerpc/v5/ndr.py", line 135, in __getitem__
    return self.fields[key]['Data']
  File "/usr/local/lib/python2.7/dist-packages/impacket/dcerpc/v5/drsuapi.py", line 603, in __getitem__
    return ''.join([chr(i) for i in self.fields[key]])
ValueError: chr() arg not in range(256)
asolino commented 7 years ago

Hey @ThePirateWhoSmellsOfSunflowers.. don't be sorry you're being very helpful :)

All right.. so looks like there's something going on at the DCERPC level. I would be awesome if we can get to know what's in self.fields[key] that is returning a value > 255.

Would you mind adding the following code before this line: print 'DATA: ', repr(self.fields[key])

and send a single result (when it's failing) here?

thanks!

ThePirateWhoSmellsOfSunflowers commented 7 years ago

I have to censor some information, but :

DATA: [67, 78, 61, 42, 42, 42, 42, 42, 42, 42, 42, 42, 32, 80, 97, 119, 101, 322, 44, 79, 85, 61, 42, 42, 42, 44, 79, 85, 61, 42, 42, 42, 42, 42, 44, 79, 85, 61, 42, 42, 42, 42, 42, 61, 42, 42, 42, 42, 42, 42, 42, 44, 68, 67, 61, 42, 42, 42, 42, 42, 42, 44, 68, 67, 61, 42, 42, 42, 42, 42, 42, 42, 44, 68, 67, 61, 42, 42, 42, 42, 42, 0]

The result should be something like

'CN=********* Paweł,OU=***,OU=*****,OU=*****=*******,DC=******,DC=*******,DC=*****\x00'

where 322 is ł in Paveł

I'm pleased to help you, impacket rocks ! :smile:

asolino commented 7 years ago

You rock @ThePirateWhoSmellsOfSunflowers!.. That was super helpful.

Can you please checkout impacket again and test it with the latest commit? If it solves the issue, then we will need to look for similar situations elsewhere in the code.

let me know!

ThePirateWhoSmellsOfSunflowers commented 7 years ago

It works ! 

[+] Calling DRSGetNCChanges for CN=NAME Michał,OU=XXX,OU=XX,OU=XX,OU=XX,DC=XXX,DC=XX,DC=XX 
[+] Entering NTDSHashes.__decryptHash
[+] ValueError exception on [REDACTED, 322, 44, REDACTED, 0]
[+] Switching to unichr()
[+] Decrypting hash for user: CN=NAME Michał,OU=XXX,OU=XX,OU=XX,OU=XX,DC=XXX,DC=XX,DC=XX
DOMAIN\NAME:486764:YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY:ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ::: (status=Enabled)
[+] Leaving NTDSHashes.__decryptHash

Thanks again @asolino ! :+1: