search

thewca / tnoodle

Development for the official WCA scramble server
https://www.worldcubeassociation.org/regulations/scrambles/
GNU Affero General Public License v3.0
400 stars 94 forks source link

Passwords can be cracked easily #198

Open vojtechkral opened 9 years ago

vojtechkral commented 9 years ago

Hi. I'm not sure if you're aware of this, but ZIP archive encryption is, well, shite. It suffers from the known plaintext attack.

TL;DR: Any TNoodle-generated password-protected zip file can be cracked in a couple of minutes.

Longer version: To crack a TNoodle zip file, all the attacker needs is to use TNoodle to create another zip file with the same competition name and use it for a know plaintext attack. The zip file the attacker creates must not be password protected and it doesn't have to contain the same set of events as the zip file to be cracked. All that matters is that the competition name is same.

This creates and identical file in both the original encrypted zip file and the attacker's zip file. It's the html file (ie. <competition name>.html). This file can then be used to conduct the attack.

I've tried this on my machine using the pkcrack utility:

$ time ./pkcrack -C TramtariaOpen.zip -c TramtariaOpen.html -P myzip.zip -p TramtariaOpen.html -d cracked.zip
Files read. Starting stage 1 on Fri Nov 20 00:12:43 2015
Generating 1st generation of possible key2_298 values...done.
Found 4194304 possible key2-values.
Now we're trying to reduce these...
Done. Left with 28541 possible Values. bestOffset is 24.
Stage 1 completed. Starting stage 2 on Fri Nov 20 00:12:54 2015
Ta-daaaaa! key0= 101e707, key1=8da3970f, key2=60f27edf
Probabilistic test succeeded for 279 bytes.
Strange... had a false hit.
Strange... had a false hit.
Stage 2 completed. Starting zipdecrypt on Fri Nov 20 00:27:45 2015
Decrypting pdf/3x3x3 Cube Round 1.pdf (47baea80157c5a772e5050b1)... OK!
Decrypting txt/3x3x3 Cube Round 1.txt (57b856508580b941f3d750b1)... OK!
Decrypting TramtariaOpen.json (b619de8dd26ae8dccf7a50b1)... OK!
Decrypting TramtariaOpen.jsonp (56bdaaa55eaeb6c2632b50b1)... OK!
Decrypting TramtariaOpen.html (9bca3cddeaa5749400d450b1)... OK!
Decrypting TramtariaOpen.pdf (99253c5a6b0055e7ace750b1)... OK!
Finished on Fri Nov 20 00:27:45 2015
./pkcrack -C TramtariaOpen.zip -c TramtariaOpen.html -P myzip.zip -p  -d   900,14s user 1,98s system 100% cpu 15:01,79 total

As you can see, it took 15 minutes to crack the archive. TramtariaOpen.zip is the zip file that I "stole" and want to crack. I don't need to know the password to this file. myzip.zip is the plaintext zip file I've created using TNoodle. cracked.zip is the resulting decrypted zip file, it contains all the juicy intel unproctected for me to read.

Note that the time to crack the file is not dependent on the password length or the character set used for the password. All generated zip file are equally vulnerable regardless of how strong a password you use.

To solve this problem, an archive format with a better encryption algorithm would have to be used, such as for instance, 7z (which uses AES as far as I know).

I'll leave it to you to judge how serious this vulnerability is, but I'll re-etiretate that as of now the password protection feature of TNoodle is pretty much as good as useless.

lgarron commented 9 years ago

To solve this problem, an archive format with a better encryption algorithm would have to be used, such as for instance, 7z (which uses AES as far as I know).

Yeah, but how does it use AES?

Delegates should be using discretion about whom they send .zip files (even if encrypted), and I suspect we would probably prefer the compatibility of .zip to ideal security. We need to trust competition staff with access to scrambles during the competition, anyhow. @jfly, wanna take this to the Board?

vojtechkral commented 9 years ago

@lgarron

Yeah, but how does it use AES?

CBC mode and a reasonably designed KDF. See this thread on security SE.

Delegates should be using discretion about whom they send .zip files (even if encrypted), and I suspect we would probably prefer the compatibility of .zip to ideal security. We need to trust competition staff with access to scrambles during the competition, anyhow.

Sure, that I understand.

jfly commented 9 years ago

Sorry I missed this issue, @vojtechkral, there was an issue with my email filtering that I just fixed.

Thanks for reporting this! It's good to know the zip encryption is not secure. As @lgarron said, one of the driving aspects of TNoodle is for things to work everywhere. In fact, password protected zip files are kind of annoying for delegates to use, as you can't open them on stock android or iOS. The real "security" comes from guarding access to the zip file.

That all said, the illusion of security is worse than no security at all. @lgarron, @vojtechkral, what do you think of removing zip passwords entirely?

lgarron commented 9 years ago

what do you think of removing zip passwords entirely?

They can still be convenient for access management and password escrow with staff you trust. I'd at least ask Delegates before removing the feature.

vojtechkral commented 9 years ago

It just occured to me that PDFs support encryption too. AES-128 is supported since PDF1.6/Acrobat7.

I've created an example PDF encrypted this way in case you want to try if the encryption is supported in your PDF reader:

http://kral.hk/KWy http://kral.hk/KWy

For me, so far this worked on all my devices/OSes...

EDIT: Oops, forgot to mention the password: 1234

jfly commented 8 years ago

Oh, neat! I am pretty sure the last time I tried this, Chrome's PDF reader didn't support passwords.

Protip, we actually do have support for PDF encryption (activated by holding shift while clicking the Scramble! button, but delegates had better not starting using it, or else they won't get JSON (#189 is one clever idea for working around that.) Here's the backend code that ultimately sets the encryption using iTextPDF:

totalPdfWriter.setEncryption(password.getBytes(), password.getBytes(), PdfWriter.ALLOW_PRINTING, PdfWriter.STANDARD_ENCRYPTION_128);

I'm guessing from http://api.itextpdf.com/itext/com/itextpdf/text/pdf/PdfEncryption.html that we should probably change that PdfWriter.STANDARD_ENCRYPTION_128 to PdfWriter.AES_128 or PdfWriter.AES_256?

Here's an example with password 1234. Scrambles for 2015-12-06.pdf

vojtechkral commented 8 years ago

@jfly You can use AES-256, but it has been added in much later PDF format version, so the compatibility might be worse. Your example is AES-128 or AES-256?

jfly commented 8 years ago

Neither, that's with our current code, which does the PdfWriter.STANDARD_ENCRYPTION_128 I copy pasted. I have no idea what that does.

vojtechkral commented 8 years ago

@jfly Oh, that's probably RC4. Try AES-128 :)

lgarron commented 8 years ago

@jfly Oh, that's probably RC4. Try AES-128 :)

"AES-128" doesn't tell us much here, either. :-P Do you know how good the RC4 implementation in PDF is? We only need a few days of forward security (and password brute-forcing is likely to be easier if anyone does ever get their hands on a PDF.)

jfly commented 8 years ago

Looks like we're using itextpdf 5.3.0: https://github.com/cubing/tnoodle/blob/33d406b831714a90cfab135cac4660025d1c9ef4/lib/itextpdf-5.3.0.jar. I can't tell when AES-128 was added, but it certainly looks like it's not present in ours =(

This is where I sign off. Someone who is motivated to upgrade our iTextPDF is more than welcome to take a dive.

vojtechkral commented 8 years ago

@lgarron Ok, sorry, according to PDF reference, they use AES-CBC, 128bit block and 128bit random IV. As far as I know, there are no grave security problems with PDF/AES-128. See also this blog post. I honestly don't know how long it would take to crack the PDF with RC4 but it's considered pretty insecure and I don't think there are any gains (such as compatibility) in using it over AES-128.

@jfly I don't think you need to update, I've downloaded your JAR and it seems to contain:

public class com.itextpdf.text.pdf.PdfEncryption {
  public static final int STANDARD_ENCRYPTION_40;
  public static final int STANDARD_ENCRYPTION_128;
  public static final int AES_128;
  public static final int AES_256;
  // ... snip
}

Not sure what the problem is but it seem AES_128 is there...

jfly commented 8 years ago

Ah, good catch. Perhaps they moved it from PdfEncryption.AES_128 to PdfWriter.AES_128 at some point. With the following code:

*** ScrambleRequest.java ***

1065         if(password != null) {
1066             totalPdfWriter.setEncryption(password.getBytes(), password.getBytes(), PdfWriter.ALLOW_PRINTING, PdfEncryption.AES_128);
1067         }

I get the following exception:

java.lang.IllegalArgumentException: No valid encryption mode
    at com.itextpdf.text.pdf.PdfEncryption.setCryptoMode(PdfEncryption.java:198)
    at com.itextpdf.text.pdf.PdfWriter.setEncryption(PdfWriter.java:2042)
    at net.gnehzr.tnoodle.server.webscrambles.ScrambleRequest.requestsToPdf(ScrambleRequest.java:1066)
    at net.gnehzr.tnoodle.server.webscrambles.ScrambleViewHandler.wrappedService(ScrambleViewHandler.java:236)
    at net.gnehzr.tnoodle.server.SafeHttpServlet.service(SafeHttpServlet.java:41)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:45)
    at winstone.ServletConfiguration.execute(ServletConfiguration.java:249)
    at winstone.RequestDispatcher.forward(RequestDispatcher.java:335)
    at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:378)
    at net.gnehzr.tnoodle.server.HtmlInjectFilter.doFilter(HtmlInjectFilter.java:40)
    at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
    at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
    at org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:176)
    at org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:145)
    at org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.java:92)
    at org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:389)
    at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
    at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
    at winstone.RequestDispatcher.forward(RequestDispatcher.java:333)
    at winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:244)
    at winstone.RequestHandlerThread.run(RequestHandlerThread.java:150)
    at java.lang.Thread.run(Thread.java:745)
vojtechkral commented 8 years ago

@jfly Yup, I can reproduce the exception, no idea how to fix it though :-/

Eyremba commented 6 years ago

To solve this problem, an archive format with a better encryption algorithm would have to be used, such as for instance, 7z (which uses AES as far as I know).

The normal .zip format supports also AES, for example in 7zip, WinRAR, PeaZip,...

gregorbg commented 4 years ago

Please see #463 for an interesting finding: With the newer zip4j versions, switching encryption algorithms literally is a one-liner, but most native archiving programs (that are pre-installed with your OS of choice) do not support stronger AES encryption when trying to deflate the ZIP archive.

We need to reach a general consensus on whether we want to require Delegates to download and install third-party archiving tools for using TNoodle.

Right now, with the "minimal dependencies" approach, this suggestion is impossible to implement.

Dhruvacube commented 1 year ago

Please see #463 for an interesting finding: With the newer zip4j versions, switching encryption algorithms literally is a one-liner, but most native archiving programs (that are pre-installed with your OS of choice) do not support stronger AES encryption when trying to deflate the ZIP archive.

We need to reach a general consensus on whether we want to require Delegates to download and install third-party archiving tools for using TNoodle.

Right now, with the "minimal dependencies" approach, this suggestion is impossible to implement.

if "minimal dependencies" approach is required then why not create implicit archiving program in TN Nooodle