AES Decryption on iPhone irregular produces Error: Malformed UTF-8 data

[GoogleCodeExporter]

1. I took a ca. 16k size encrypted file and decrypted it these way
   CryptoJS.AES.decrypt(encrypted_string, sPassword).toString(CryptoJS.enc.Utf8);

2. in Firefox 19 (and lower), Safari its working well but on mobile Safari and 
Webapp (on different iPhones) I get the  
   exception

3. But if I put it in many try/catch blocks sometimes it's working on "a" try 
and often on "c" try
   e.g. like this example: http://pastebin.com/ZEry2LMS

   This isn't nice but so it's actually working and not break my application

In all my testcases I made use of the same encrypted string.

I made use of CryptoJS v3.1.2 on iOS Device (iPad not tested) and expect to get 
JSON String representation but mostly get Malformed UTF-8 Data exception 

A not so nice workaround was to put many try block until I try is successfull 

Original issue reported on code.google.com by bernhard...@googlemail.com on 27 Feb 2013 at 12:19

[GoogleCodeExporter]

Hi,

I have the same identical issue on ios6 ipad. 

I believe this is due to a bug in the javascript interpreter on ios.

I tracked down the problem to sporadic differences in output from the base64 -> 
wordarray decoder on ios. The output varies - sometimes it is correct but 
sometimes it will decode a single byte incorrectly. This is enough lead to 
failure to decrypt the data correctly. 

here's the problematic block:

for (var i = 0; i < base64StrLength; i++) {
    if (i % 4) {
        var bits1 = map.indexOf(base64Str.charAt(i - 1)) << ((i % 4) * 2);
        var bits2 = map.indexOf(base64Str.charAt(i)) >>> (6 - (i % 4) * 2);
        words[nBytes >>> 2] |= (bits1 | bits2) << (24 - (nBytes % 4) * 8);
        nBytes++;
    }
}

I found that 

1. if you attempt to log the values of bits1 and bits2 the problem goes away
2. if you break the bits1 | bits2 expression into a separate line the problem
    goes away, i.e do 

var bits_combined = bits1 | bits2;
words[nBytes >>> 2] |= bits_combined << (24 - (nBytes % 4) * 8);

so i think there is some problem evaluating | as a temporary. uh oh.

I have attached 

1. a patch with the above
2. my test harness, my logging patches to cryptojs, and some logs illustrating
   the problem

Original comment by iterat...@gmail.com on 2 Mar 2013 at 1:27

Attachments:

• base64.patch
• logpatch.patch

[GoogleCodeExporter]

log attachment

Original comment by iterat...@gmail.com on 2 Mar 2013 at 1:28

Attachments:

• log_ios11.txt

[GoogleCodeExporter]

log attachment

Original comment by iterat...@gmail.com on 2 Mar 2013 at 1:28

Attachments:

• log_chrome11.txt

[GoogleCodeExporter]

test harness

Original comment by iterat...@gmail.com on 2 Mar 2013 at 1:28

Attachments:

• index.html

[GoogleCodeExporter]

I am having the same issue in any iOS device version 6.1.3.
The interesting thing is if I connect to the device and watch the code in the 
developers console, it never fails. 

If I disconnect from the developers console and execute, it fails every time on 
files 4MB or larger (I haven't tested to find the failure size). Even if 
running in an infinite do loop trying to catch a good conversion.

It does seem to work every time if files are in the < 300KB.

I compiled with the patch in place using SIMPLE_OPTIMIZATIONS and 
WHITESPACE_ONLY with pretty print on and off, and have same issue.  Everything 
works fine in Chrome on iOS devices.

    for(var i = 0;i < base64StrLength;i++) {
      if(i % 4) {
        var bits1 = map.indexOf(base64Str.charAt(i - 1)) << i % 4 * 2;
        var bits2 = map.indexOf(base64Str.charAt(i)) >>> 6 - i % 4 * 2;
        var bits_combined = bits1 | bits2;
        words[nBytes >>> 2] |= bits_combined << 24 - nBytes % 4 * 8;
        nBytes++
      }
    }

Any ideas?

Original comment by flagg.n...@gmail.com on 7 May 2013 at 11:17

[GoogleCodeExporter]

I'm having the same issue on IOS 6.1

I'm not having any luck, regardless of the file size and I have applied the 
patch.  AES decryption just isn't working.  My phonegap app runs just fine in 
the simulator and fails to decrypt when running on the actual device. 

Original comment by mbal...@gmail.com on 8 May 2013 at 1:17

[GoogleCodeExporter]

hi

running the patched js through a js compiler is likely to rearrange the 
statements and trigger the problem again.

the patch is just a workaround for something that appears to be an interpreter 
problem, i have no insight into the root cause. 

we are trying to move away from cryptojs and do this natively anyway because 
its just too slow in a UIWebView anyway and the whole UI freezes during 
decryption. 

Original comment by iterat...@gmail.com on 8 May 2013 at 1:31

[GoogleCodeExporter]

The above code snippet was from the compiled code - I included it in case there 
was something the compiler changed or I got wrong. Included below just in case:

    for(var i = 0;i < base64StrLength;i++) {
      if(i % 4) {
        var bits1 = map.indexOf(base64Str.charAt(i - 1)) << i % 4 * 2;
        var bits2 = map.indexOf(base64Str.charAt(i)) >>> 6 - i % 4 * 2;
        var bits_combined = bits1 | bits2;
        words[nBytes >>> 2] |= bits_combined << 24 - nBytes % 4 * 8;
        nBytes++
      }
    }

I've reported the problem to Apple. Again, everything worked fine until iOS 
6.1.3.  

The thing I find most confounding is the fact aes.js works flawlessly in Chrome 
in iOS 6.1.3 (iPhone, iPad, haven't tested iPod). It also works flawlessly on 
an iOS device with an active debugging console open. And it fails reliably in 
iOS 6.1.3 if the device does not have an open console. 

I did write the wrong file sizes though. It fails when a file is bigger than 
about 300KB. It seems rock solid with 10-30KB files.

I'm game for anything, what are you using as a replacement for cryptojs? 

In the interim, I've written an FAQ instructing my users to use Chrome rather 
than Safari if they've upgraded to iOS 6.1.3 while I chase this.

Cheers,
Neil

Original comment by flagg.n...@gmail.com on 8 May 2013 at 5:44

[GoogleCodeExporter]

i haven't switched it yet, but im just going to just use openssl from
obj-c. we are a cordova app so can easily delegate to native async methods
which will also not block the ui theread.

some possibilities:

1. i probably haven't really isolated the problematic line; thats just what
worked for me. i located the problem by just putting in a lot of logging
into crypto and checking what values were changing that shouldn't have
been. you could do the same
2. the prominence behaviour might vary by ios version
3. its probably due to non-initialization of a variable in the interpreter
which means the behaviour is undefined and that contributing factors are
almost impossible to guess without knowing the interpreter code
4. there are possibly other lines in crypto which are triggering the
behaviour - i didnt look around

Original comment by iterat...@gmail.com on 8 May 2013 at 5:58

[GoogleCodeExporter]

Thanks for the info. I'll take a look into cordova. 

The problem didn't appear to exist in the previous version of iOS.  Since 
aes.js performs flawlessly on larger files in so many other mobile browsers, 
works in the Apple debugger console, that points toward the problem being an 
interpreter issue.

Quick correction on file sizes: Since iOS 6.1.3 mobile Safari regularly fails 
to decrypt files of significant size (400KB+).

I will wait to hear from Apple on the bug report. Not worth spending weeks 
chasing since my users have a large number of other viable options.

Test results:
Decryption of files up to 1 MB works flawlessly using Google Chrome on iPhone 5 
and iPad 2 hardware.
Issue is not present in Safari 6.0.4, Chrome Version 26.0.1410.65, Firefox 
20.0, Internet Explorer 7, 8, 9, and 10, default browsers on Android devices < 
16 months old (phones, Kindle Fire etc.), and Windows 8 mobile.

Original comment by flagg.n...@gmail.com on 8 May 2013 at 2:53

[GoogleCodeExporter]

If a large file is correctly decrypted (while mobile Safari is connected to 
debugger) and the is then encrypted and written to localStorage, mobile safari 
can read and decrypt the variable from localStorage.

mobile Safari using aes.js will successfully encrypt and decrypt 1MB files if 
file is read from and written to localStorage in stand alone mode.

Writing web content to localStorage first then attempting to decrypt from 
localStorage variable does not resolve the stand alone issue. Byte count of 
file to be decrypted is identical when iOS device is stand alone or has a 
developers console open. Both before and after writing to localStorage.

Original comment by flagg.n...@gmail.com on 8 May 2013 at 4:07

[GoogleCodeExporter]

While writing a hunk of sample code to demonstrate the issue, I found a work 
around (a bit clumsy, but it works).
The concise issue:
Using CryptoJS JavaScript as encryption library.
If mobile Safari in iOS 6.1.3 downloads an AES encrypted file of more than 
roughly 200KB-300KB, the decryption fails 100% of the time.
(Decryption is 100% successful if using a locally stored AES encrypted files, 
or if the iOS device is connected to the Web Inspector for debugging.)

The work-around:
download aes file.
Attempt to decrypt.
On failure to decrypt:
  repeat above until successful.

Just re-sending the original downloaded file back for decryption does not work. 
The file must be re-downloaded.

This has proven 100% successful on iPhone 4, iPhone 5, iPad 2, The New iPad.
Will test on iPhone 4S as soon as I upgrade to 6.1.3.

Once mobile Safari has successfully decrypted a file of this size, all 
following files will decrypt first time through.

My iPad 2 requires 2 downloads, the second successfully decrypting.
My iPhone 5 requires 3 downloads, the third successfully decrypting.

sample code and data file attached.

Original comment by flagg.n...@gmail.com on 9 May 2013 at 5:38

Attachments:

• debug.html
• sampleenc.txt

[GoogleCodeExporter]

As written in initial bugreport, I found a solution working better with all my 
devices.
I implemented a block of try/catch statement 3 times and it wasn't ensured to 
get a success decoded string.

A few days after the report, I found a better way and I think it might be 
interesting for someone until the problem is solved.
I know it's dirty but on my iPhone a file of about 4Mb was successfull 
decrypted — after a while ;-) 

var bSuccess = false;
while (!bSuccess) {
  try {
    sJson = CryptoJS.AES.decrypt(sEncrypted, sPassPhrase).toString(CryptoJS.enc.Utf8);
    bSuccess = true;
  } catch(e) {
      //bSuccess=false;
    }
  }

Maybe it's usefull for someone.

Original comment by bernhard...@googlemail.com on 10 May 2013 at 5:33

[GoogleCodeExporter]

This sounds similar to an issue that almost made its way into Chrome.

CryptoJS issue was reported here: 
https://code.google.com/p/crypto-js/issues/detail?id=72
Corresponding Chrome issue was reported here: 
https://code.google.com/p/chromium/issues/detail?id=173907

If this is indeed an interpreter issue, I'd be very appreciative if someone 
with more time than I could report this to IOS and/or WebKit.

Original comment by Jeff.Mott.OR on 10 May 2013 at 6:19

[GoogleCodeExporter]

naf-101 says : "..found a work around (a bit clumsy, but it works).."

https://discussions.apple.com/thread/5022036

Original comment by Wright.H...@gmail.com on 24 Oct 2013 at 1:39

[GoogleCodeExporter]

Any news for that problem ?
I´m using that library on a project
and that problem began today,
when users try to login on my app,
sometimes the decrypt process fails with this error.

Original comment by tschle...@gmail.com on 29 Apr 2014 at 6:09