Import Compromised Password hash PS command failed to import

[bodysoda]

Hi there, I've downloaded the latest copy of password hard and unzip the txt file to my server. When I run the PS Import-CompromisedPasswordHashes -Filename C:\DFSRoots\PasswordHash\pwned-passwords-ntlm-ordered-by-hash-v4.txt I get the following error message. Can you please help?

Import-CompromisedPasswordHashes : The file contained a line that was not recognized as a hexadecimal hash. Lines must end with a new line character or colon At line:1 char:1

• Import-CompromisedPasswordHashes -Filename C:\DFSRoots\PasswordHash\p ...

• 
  + CategoryInfo          : NotSpecified: (:) [Import-CompromisedPasswordHashes], InvalidDataException
  + FullyQualifiedErrorId : System.IO.InvalidDataException,Lithnet.ActiveDirectory.PasswordProtection.PowerShell.Imp
  ortCompromisedPasswordHashes

[ryannewington]

Hi @bodysoda,

What does the first and last lines of the file look like? Have you edited this file at all, or was it downloaded straight from HIBP and unzipped?

Ryan

[bodysoda]

I downloaded the file HIBP NTLM (ordered by hash) and unzipped it. Haven't done any modifications. Can't view the files with any editors (notepad, notepad ++).

[ryannewington]

ok let me download it again and see if i can reproduce

[ryannewington]

oh just out of curiosity what did you use to unzip it @bodysoda

[bodysoda]

oh just out of curiosity what did you use to unzip it @bodysoda

Yes, I unzip it which resulted to pwned-passwords-ntlm-ordered-by-hash-v4.txt

[ryannewington]

What program did you use to unzip it?

[bodysoda]

What program did you use to unzip it?

Unzipped using 7zip (https://www.7-zip.org/a/7z1900-x64.exe)

[ryannewington]

Thanks, I will make sure I test with the same version and report back soon

[bodysoda]

oh just out of curiosity what did you use to unzip it @bodysoda

Yes, I unzip it which resulted to pwned-passwords-ntlm-ordered-by-hash-v4.txt

SHA-1 Has of 7zip file matches to the one on the HIBP site ee7199ee2a1d8f23dd346d5b1fb2255e1ed8de8a

[bodysoda]

Thanks, I will make sure I test with the same version and report back soon

Ok, Thanks appreciate the quick response and support.

[ryannewington]

@bodysoda, No luck reproducing it on my side i'm afraid. Let's check a few things.

1. Can you let me know the size of your file. Does it match mine?

2. Does the error appear immediately, or after the hash import process has started?

3. If you run type pwned-passwords-ntlm-ordered-by-hash-v4.txt | more Are the first few lines as appear below?

   00000001F4A473ED6959F04464F91BB5:4
   0000000F23B59311F4FFB01D6D620487:2
   000000113BDEB707C98A8234826BF788:4
   000000128830292D92FA6B226EEC986B:3
   00000015B1284879951DC072C80735DC:3
   0000001991F12B30E3B00E7CCED2ADFB:1
   0000001FE8FBE6BE79FC5A0D39CDFD68:4
   0000002ACC41B93A9CFB5E227503E6F2:3
   000000307E7F80DC6BB8572C0DA9A8BA:3
   000000393819A00A1DD98DCCA628B8F7:2
   0000003B1EFB160FD6E6F709D39FBEBC:2
   00000040A662EE8927F973E3656DB6F0:4
   0000004A130EF7A7AF23CB99F7D9F51B:1
   0000004FC1E5E3102E408CD65BA3D576:6
   00000058E7871D0E2282B208017E67B3:2
   0000005EC420AD1C63F832990A606162:3
   0000006C573BEB4CDDC2553839CF85D6:6
   0000007637C0A140696BA14B7097F1B3:1
   00000079BBB637214186BDB9EBCE422E:2
   00000080E5D8025A5AA187F3002012A3:1
   000000841F8C2DB95A6AD9FB20FF9CE2:1
   0000008F10B9BCB51AAB81BFAFC0EB8C:2
   000000930FD7A1A040E8B0C72A72B2EB:3
   000000952FA67C098C82A59D6816847F:2
   0000009ACE7B03FF546FE8875E382203:2
   0000009F8DC7BBBABF4E0DC1AE68D3C8:2

[bodysoda]

I don't recollection seeing my unzip txt file is 18.5GB. However I will check it when I am back in the office tomorrow.

[bodysoda]

@bodysoda, No luck reproducing it on my side i'm afraid. Let's check a few things.

1. Can you let me know the size of your file. Does it match mine?

Yes, my txt file looks exactly like your.

1. Does the error appear immediately, or after the hash import process has started?
2. If you run type pwned-passwords-ntlm-ordered-by-hash-v4.txt | more Are the first few lines as appear below?

00000001F4A473ED6959F04464F91BB5:4
0000000F23B59311F4FFB01D6D620487:2
000000113BDEB707C98A8234826BF788:4
000000128830292D92FA6B226EEC986B:3
00000015B1284879951DC072C80735DC:3
0000001991F12B30E3B00E7CCED2ADFB:1
0000001FE8FBE6BE79FC5A0D39CDFD68:4
0000002ACC41B93A9CFB5E227503E6F2:3
000000307E7F80DC6BB8572C0DA9A8BA:3
000000393819A00A1DD98DCCA628B8F7:2
0000003B1EFB160FD6E6F709D39FBEBC:2
00000040A662EE8927F973E3656DB6F0:4
0000004A130EF7A7AF23CB99F7D9F51B:1
0000004FC1E5E3102E408CD65BA3D576:6
00000058E7871D0E2282B208017E67B3:2
0000005EC420AD1C63F832990A606162:3
0000006C573BEB4CDDC2553839CF85D6:6
0000007637C0A140696BA14B7097F1B3:1
00000079BBB637214186BDB9EBCE422E:2
00000080E5D8025A5AA187F3002012A3:1
000000841F8C2DB95A6AD9FB20FF9CE2:1
0000008F10B9BCB51AAB81BFAFC0EB8C:2
000000930FD7A1A040E8B0C72A72B2EB:3
000000952FA67C098C82A59D6816847F:2
0000009ACE7B03FF546FE8875E382203:2
0000009F8DC7BBBABF4E0DC1AE68D3C8:2

Yes,I can view the contents via type command and output looks exactly as same to your output.

[ryannewington]

Thanks @bodysoda, does the error appear immediately or does it start to import rows and then fails?

Can you also confirm you OS version, PowerShell version, and .net framework version?

[bodysoda]

Thanks @bodysoda, does the error appear immediately or does it start to import rows and then fails?

The error occurs when the Powershell nearly gets to 89% ...

I will try to extract the HIBP txt file and upload to my test domain controller. I wonder if WINSCP upload is modifying the files. Standby for updates.

Can you also confirm you OS version, PowerShell version, and .net framework version?

DC is Windows 2012 R2, PS 5.1,

[ryannewington]

To see if something has modified it, you can check the sha1 hash of the text file. It should be 861ab6091e3e98d535267c5e0cba7764a0ab319b

certutil -hashfile "D:\pwnedpwds\raw\pwned-passwords-ntlm-ordered-by-hash-v4.txt" SHA1

[bodysoda]

To see if something has modified it, you can check the sha1 hash of the text file. It should be 861ab6091e3e98d535267c5e0cba7764a0ab319b

certutil -hashfile "D:\pwnedpwds\raw\pwned-passwords-ntlm-ordered-by-hash-v4.txt" SHA1

Ok, Its works .. sort off... I uploaded the HIBP txt file and PowerShell Import-compromisedpasswordhash command completed successfully. This created a folder V3 which contained sub-folder "p" and "w" on them. P sub-folder has some .dll files and w is BLANK.

I created a new GPO as mentioned on the Wiki and both domain controllers have LPP installed, HIBP store is replicated via DFS-R.

When creating a test username with password "P@ssw0rd", AD doesn't block creation with weak password.

PS C:\windows\system32> Import-Module LithnetPasswordProtection PS C:\windows\system32> Get-PasswordFilterResult -Password P@ssw0rd -username "test_user" -Fullname "Test User" Approved

What am I doing wrong?

[ryannewington]

Hi @bodysoda,

the 'p' folder is the compromised password store, and 'w' is for banned words. So this is as expected if you haven't added any banned words.

The machine you are running the PowerShell cmdlet on must have 1) access to the store, and 2) the LPP GPO applied to it to know which rules to process.

You can use the Open-Store cmdlet if the store is in a location other than where you specified when you installed the LPP module on that machine.

[ryannewington]

@bodysoda do you need any further assistance with this?

[eantoniope1]

Ryan,

I'm also getting this error when I try to import the list....

Import-CompromisedPasswordHashes : One or more errors occurred. At C:\Users\admin_eantonio\Desktop\Lithnet.ps1:3 char:1

• Import-CompromisedPasswordHashes -Filename E:\Temp\pwned-passwords-nt ...

• 
  + CategoryInfo          : NotSpecified: (:) [Import-CompromisedPasswordHashes], AggregateException
  + FullyQualifiedErrorId : System.AggregateException,Lithnet.ActiveDirectory.PasswordProtection.PowerShell.ImportCompromisedPasswordHashes

I get this error as soon as I run the ps script. Your help is much appreciated.

[eantoniope1]

The filename of the list I downloaded is pwned-passwords-ntlm-ordered-by-hash-v5.txt and it is 18.6 GB file size.

[ryannewington]

@eantoniope1 are you using the latest version of the app -> https://github.com/lithnet/ad-password-protection/releases/tag/v1.0.7143

If so, immediately after the error appears, can you type $error[0].Exception.InnerException.ToString() and paste the result here

[eantoniope1]

Yes, I installed the latest version but I still get this error. Here's the error you requested.

PS C:\Users\admin_eantonio\Desktop> $error[0].Exception.InnerException.ToString() System.IO.IOException: The process cannot access the file 'E:\Program Files\Lithnet\Active Directory Password Protection\Store\v3\p\016B.db.bin' because it is being used by another process. at System.IO.Error.WinIOError(Int32 errorCode, String maybeFullPath) at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptio ns options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost) at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share) at Lithnet.ActiveDirectory.PasswordProtection.BinaryStoreInstance.WriteStoreFile(String file, Boolean append, IEnumerable1 hashes) at Lithnet.ActiveDirectory.PasswordProtection.BinaryStoreInstance.AddHashRangeToTempStore(HashSet1 hashes, String range) at Lithnet.ActiveDirectory.PasswordProtection.Store.<>c__DisplayClass27_0.b0(KeyValuePair2 group) at System.Threading.Tasks.Parallel.<>c__DisplayClass42_02.b1() at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask) at System.Threading.Tasks.Task.<>cDisplayClass176_0.b__0(Object )

[ryannewington]

@eantoniope1 are you using a DFS-R based store? You may need to pause the replication group while you build the store. LPP is trying to write to the store, but something else has locked open its files. If you aren't using DFS-R something else must be locking these files. Maybe AV?

[eantoniope1]

I'm not using DFS-R. I will see if I can whitelist Lithnet from my AV. I will keep you posted.

[ryannewington]

It would only be needed while you build the store. It doesn't need a permanent exemption.

[eantoniope1]

Ryan,

I added the Lithnet folder in my AV exception list. Now I get this error.

Import-CompromisedPasswordHashes : Line #555278658 was not recognized as a hexadecimal hash. The line was not the expected length. The following line was invalid: NULL At C:\Users\admin_eantonio\Desktop\Lithnet-count.ps1:3 char:1

• Import-CompromisedPasswordHashes -Filename C:\IT\Lithnet-PasswordProt ...

• 
  + CategoryInfo          : NotSpecified: (:) [Import-CompromisedPasswordHashes], InvalidDataException
  + FullyQualifiedErrorId : System.IO.InvalidDataException,Lithnet.ActiveDirectory.PasswordProtection.PowerShell.ImportCompromisedPasswordHashes

[eantoniope1]

Ryan,

I also checked the Lithnet folder - E:....Lithnet\Active Directory Password Protection\Store\v3\p folder... it is now 27.8 GB. It grows everytime I run the import hash powershell script. Is this normal size?

Should I delete everything and start over?

[ryannewington]

Youll need to redownload the pwned password list from haveibeenpwned. There was an error in the source file that resulted in NULL vales in the file. The file has since been fixed.

You can delete all the files in the 'p' folder to reclaim that space. It should be about 7gb once it has been properly built.

[eantoniope1]

I will re-download the source file and will keep you posted. Thank you.

[eantoniope1]

Ryan,

the new source code is now working! Thank you so much for your help.