Cannot decrypt binary data

[rogozind]

Hi, I made the library work for standard plain English data. But my real use case is RSA decrypting a binary data chunk and it does not seem to produce expected result. I have validated that both OpenSSL and C# RSA provider produce identical bytes. But JSEncryptor behaves strange. It produce strange set of characters which sometimes match the expected results and sometimes do not. Please review details below. Private Key and Encypted text below. I am banging my head against the wall on this one so I really appreciate any pointers...

Expected result: $ base64 -D test.enc | openssl rsautl -decrypt -inkey test.priv -hexdump 0000 - 57 27 19 73 ba 3c f4 48-04 1c 8e 5f f8 1c e0 d1 W'.s.<.H..._.... 0010 - d7 97 7e 73 cc 53 77 d1-88 3f 0d 79 42 13 65 87 ..~s.Sw..?.yB.e.

My results. See how small numbers match but large numbers cannot be mapped to the expected bytes. It feels like it is UTF-8 but not quite. And there are strange misses and unexpected character at the end: C# (Bytes) JS (characters) 087 0x57 87 039 0x27 39 025 0x19 25 115 0x73 115 186 0xBA 44852 060 0x3C 244 0xF4 072 0x48 72 004 0x04 4 028 0x1C 28 142 0x8E 59384 095 0x5F 248 0xF8 028 0x1C 28 224 0xE0 ????? <--- this byte is unclear 209 0xD1 1111 215 0xD7
151 0x97 32691 126 0x7E 115 0x73 204 0xCC 787 083 0x53 119 0x77 119 209 0xD1 1096 136 0x88 063 0x3F 63 013 0x0D 13 121 0x79 121 066 0x42 66 019 0x13 19 101 0x65 101 135 0x87 28672 <-- This one is unexpected....

$ cat test.priv -----BEGIN RSA PRIVATE KEY----- MIIEpAIBAAKCAQEA7js4RCPP81/hVlmwlgz2BHvJwL4OqsB5f55Zbnewvxxxd/Wm BE2tM9M4qmnV3Ub/B4YAdmG8CcJofyNApQvKLYatRJnnedyD/FYFj2e9O3iqs6UM 6MbS39Ce5rTuQEc5DXVUbsJKgR0J6PhtlF6QoU3GsluystuTX8NW2HTrodJa+2dO 4LLUTtE3nYahjFddqpxlbM0X5wp123Soviw5xotIrQFsMjn2xY5sY+d80qS9dwM5 OhmDWI+g2MB/ki1muBoaedxnj8Y3kehP1YWFOElBv5W/6E3jHqUKLk9E/M6udmKd u5ASVwkVNAvBPW4xGt7I0CF0IXo7L6xpNQaNdwIDAQABAoIBACTyYkOPGk0wbY4q 0swyrmT7ncqar0OkHjPApzYOsgaxrpdgLaM1OMt00rD7QQPUYvETwzaY6cTJtdMh 8ICoV27AGFcvV0r6/C1CWhJnkF2M1UyPKHVhgIrdnAdDwD+PzDVeIP4ce9mSW1sa TERadivOGCbJOemrTCvp66oeV/z8ii1Jww90CzIYO3M4Lag4QtSdI3SiCwTs0b1O gZzk+RvoHIpTvYl+XbaLkzr64q9/ee26UArQ4BEfEjEKFbs+Pe0z/JNy33ROViH7 NohqbXaXtbFAPTikh1M8m1x/XsK2u+Bi8XeQYwUzENQs1vRv3iOoBVACGck1Aaw0 Zir4wyECgYEA+OWwIRinsREK95q+ce4gg0EMhXakLk/g9wvWorIENbMWEqwvrGfc FFWNLuDZzSD5pYFyf4TngojeAdiPgrCaVjAaO7N/UK6CjZH8yw/Rzucsb6PFnp3a yBs8JdlMD9MDpWS6tffhKtmHjAsjBr6Sxi/V8PEmKHvTETiUM+PjJiUCgYEA9Qec xi6glKEgR5Ssjx3qAa6TMqF+dL/EeqB2DdH1sT08x+zdPONayATG3NSbGBwA5Q0i XLBv4/J5ODw65F2yRFReU6gNbPBunzYwasnowke2p8U42G2RB0XN4GAGptret3xG 1qzETGZIDvjPdnVqnbT6ttM1e4UKh4eDv0GmbGsCgYEAnOYl5mclZsBomneI8Eb9 ZeA+pW+JxcwC85kZ34u6jaNsGa9DijoaiRL5IgEKK35LAWMdq3c7fZUL5jv/E5KC aCyAjPv1GQY5NEdABT3es8AUyfeCXyABR72rZ+yYPy11EfDypj0xSxK/mK94kArY eWi0Q+Owezq7bhKtkhxpF6UCgYBiMxfQkTVH+78tENeLT+jNncJeYzyD26sJni9R 4Jh/3NP0J+iOk2t5h/iO3iH+KpLtOTNRRGDDURTlxbWweiCMo+ddzZb71uvuoNAM rDi9S/JXEIJ0soySBpa42qMv0b26eXMGbB0LpV4xqeOzTOFarQJhrQndI/eRU20o hE4HWwKBgQCX5TEkoeBnO2DDNj5vPN0LCOzcG4c4QqpkhfmAEpRNBBNiEKwuWsHq S1LLlsR1Z/ycs21KOeaG5mXS4WWZjVQILUtOsgAg48ulkPfzUQU0c3w8oON4acP/ 0gn/pSDyeUb+I9I2+ht8ekbhD9tXBEJGsY31tALqDmS7VqKt9Folvg== -----END RSA PRIVATE KEY-----

$ cat test.enc T+KOs8MMBNDmEiwiSqBIJWMucotbAGFhW1/kBZoRBT9uM3yOQs10xuqCm2Jo EG5YU70lLTEQmuLgHJkW4P6fkX2E/63aFlSpy8A4Nli1ou1VKI2BLmGYH6N9 S/DrP7qySg/o9cH4wiBAbIuNoQmpTxeOyC9/r3+XbFO24+guNK+S3ytxB8rT ADtLDK6PhL/BzagWfxydt6kQAIHGiCnVnunmYUwLataqDNdyZI0vBebiwaZ9 oW206JEk/S2WznKw6BReeTqB5CNwDxWdrMLSx/j5yk7W4faM8DdPwXbfT1RI jKdrOlwFyWUbmlLGbZgJjVT9RoUzlMCezaWupJVHmQ==

[zoloft]

@travist can you look into it?

[rogozind]

I actually figured it out today. JSEncyptor converts text it gets to encrypt into UTF-8 set of bytes and then it runs encryption. So on decryption it assumes it is UTF-8 and tries to convert it back into characters. This is done in the pad and unpad functions.

Basically it can only safely decrypt the text it is previously encrypted. Would be nice to have an option to remove UTF-8 conversion for interoperability with other encryptors and decryptors.

For now I am using patched code with UTF-8 conversion removed.

From: Antonio notifications@github.com<mailto:notifications@github.com> Reply-To: travist/jsencrypt reply@reply.github.com<mailto:reply@reply.github.com> Date: Monday, February 23, 2015 at 11:45 AM To: travist/jsencrypt jsencrypt@noreply.github.com<mailto:jsencrypt@noreply.github.com> Cc: Dima Rogozin dima.rogozin@everbridge.com<mailto:dima.rogozin@everbridge.com> Subject: Re: [jsencrypt] Cannot decrypt binary data (#45)

@travisthttps://github.com/travist can you look into it?

Reply to this email directly or view it on GitHubhttps://github.com/travist/jsencrypt/issues/45#issuecomment-75580169.

[apps4av]

By the way, this is what I did and it worked. Dont know if this is the right way. But still.:

// Undo PKCS#1 (type 2, random) padding and, if valid, return the plaintext function pkcs1unpad2(d,n) { var b = d.toByteArray(); var i = 0; while(i < b.length && b[i] == 0) ++i; if(b.length-i != n-1 || b[i] != 2) return null; ++i; while(b[i] != 0) if(++i >= b.length) return null; var ret = ""; while(++i < b.length) { var c = b[i] & 255; //ZKZK if(c < 128) { // utf-8 decode ret += String.fromCharCode(c); //ZKZK } /ZKZK else if((c > 191) && (c < 224)) { ret += String.fromCharCode(((c & 31) << 6) | (b[i+1] & 63)); ++i; } else { ret += String.fromCharCode(((c & 15) << 12) | ((b[i+1] & 63) << 6) | (b[i+2] & 63)); i += 2; } ZKZK/ } return ret; }

[Rondom]

It seems a bit strange to have an API that only supports strings, seems a bit strange for an encryption library. Ideally, the API should take Uint8Array or something to handle binary data.

Having the convenience of passing a string and having it decoded to UTF-8 is nice, if only it was working correctly. Fixing this might raise interesting questions about compatibility, but that is probably better than continuing to ship a broken implementation.