search

facebookarchive / conceal

Conceal provides easy Android APIs for performing fast encryption and authentication of data.
http://facebook.github.io/conceal/
Other
2.96k stars 431 forks source link

NativeGCMCipherException on Android 8.1 #184

Open alessandrojp opened 6 years ago

alessandrojp commented 6 years ago

Hi,

I have installed the preview of Android 8.1 api 27 and it is failing to decrypt getting the error below: The same process doesn't happen on on 8.0 and prior.

Crypto#decrypt com.facebook.crypto.cipher.NativeGCMCipherException: The message could not be decrypted successfully.It has either been tampered with or the wrong resource is being decrypted.

I know that is still in a preview stage, but maybe you could take a look to check what is the reason.

Thank you.

memichel commented 6 years ago

I can reproduce this exception too but only after save some input and update from Android 8.0 to 8.1 (I don't test the other cases, previous of 8.0). Then, in Android 8.1, cleaning application data allow to use again conceal. Is there something change to cipher / decipher data on new Android release ?

memichel commented 6 years ago

No news on this crash ? Because it's still present with the latest Android DP. Below, you can see my latest tests:

Android 7.1

Update to 8.1 via OTA (build OOP6.171019.012)

Is somebody observe the same behaviour ?

alessandrojp commented 6 years ago

What is the library version that you are using? I have fixed this issue by updating from 1.0.7 to 1.1.3

memichel commented 6 years ago

I'm already using 1.1.3...

alessandrojp commented 6 years ago

Are you using 128 or 256 bits to decrypt? After updating it to 1.1.3 and changing the implementation to encrypt it using 256 bits, I could decrypt it without any problem.

It seems to be an issue in the library and not in the OS.

psycholic4 commented 6 years ago

I have a same problem. Updating library cannot resolve this problem because I need to decrypt a encrypted text (already made from 7 or 8.0) in 8.1. (as mentioned above by @memichel)

com.facebook.crypto.cipher.NativeGCMCipherException: The message could not be decrypted successfully.It has either been tampered with or the wrong resource is being decrypted.
     at com.facebook.crypto.cipher.NativeGCMCipher.decryptFinal(NativeGCMCipher.java:105)
     at com.facebook.crypto.streams.NativeGCMCipherInputStream.ensureTagValid(NativeGCMCipherInputStream.java:109)
     at com.facebook.crypto.streams.NativeGCMCipherInputStream.read(NativeGCMCipherInputStream.java:90)
     at com.facebook.crypto.streams.NativeGCMCipherInputStream.read(NativeGCMCipherInputStream.java:75)
     at com.facebook.crypto.Crypto.decrypt(Crypto.java:146)
cre8ivejp commented 6 years ago

In my case I encrypted the file again using the version 1.1.3 to solve this. But for you seems to not be an option. What was the version used to encrypt in 7.1 and 8?!

psycholic4 commented 6 years ago

Our product used 1.0.7 before. Re-encrypt is not an option because we don't know original text without decrypting it.

cre8ivejp commented 6 years ago

I am afraid this will not be fixed anytime soon. One thing you could do is upload the encrypted file to your server, so you could decrypt it using bouncy castle (https://www.bouncycastle.org) and encrypt it again to make it work with the version 1.1.3 So the client side would be able to decrypt it without any problem.

I know this because I encrypt the log file on the client side and I decrypt it using bouncy castle on server side to see the logs.

Either way you will have to do some work to make this work.

helios175 commented 6 years ago

You can try a newer version and still use the same format. If you call AndroidConceal.get().createCrypto128Bits(keyChain) it will use the same format as the default one from 1.0.7. In fact there are only two:

0x01 0x01 -> 128 bits key encrypted (IV + cipher data + tag)
0x01 0x02 -> 256 bits key encrypted (IV + cipher data + tag)

Then you can call decrypt(...) with the complete byte[] or an input stream. I will check the log of commits to see if there was some bug that could be caused by differences in how OS behave (like always retrieving data in certain block sizes or not).

UPDATE: If the data was encrypted with 1.0.x then you need to pass Entities created with Entity.utf16(...). The entity is simply some 'id' for the data, and it's part of the integrity verification (so a bad entity might make it fail). In Conceal 1.0.x the entity used UTF-16 to transform the text argument into bytes. Problem is UTF-16 can be little endian (LE) or big endian depending on platform. So for 1.1.x and on we use now UTF-8.

In short:

Just guessing:

what happens if UTF-16 encoding changed from 8.0 to 8.1? (at least for the method Entity used). In that case you could try forcing the conversion. Inherit Entity and override getBytes method to return exactly what you want: "yourentitystring".getBytes("UTF-16BE") or "yourentitystring".getBytes("UTF-16LE")

bhargman commented 6 years ago

what happens if UTF-16 encoding changed from 8.0 to 8.1?

This seems to be the key. Here's what new Entity(String) would return pre-8.1: image

And here's 8.1 and up: image

So, it looks like the endianness got switched (perhaps only on some devices, but I just used AVDs). However, "hi".getBytes("UTF-16BE") or "hi".getBytes("UTF-16LE") will not work, because those methods wipe out the BOM: image

It's better to return whatever the pre-8.1 device was doing as a byte array if overriding Entity.getBytes(). For example:

@Override
public byte[] getBytes() {
    return new byte[] {-1, -2, 104, 0, 105, 0};
}
browep commented 6 years ago

any update to this? Still trying to find a smooth solution for users who upgraded from 8.0-8.1 to be able to read their old data. Don't have original file to re-encrypt.

vandac commented 5 years ago

I experienced this issue after upgrading to Android 9.

I fixed this by overriding the Entity class with two strategies. The first one uses UTF-16LE )little endian which was used up till Android 8) to read old data and UTF-8 to encode into UTF-8. The backwards compatible utf16 method does not work as on Android 9 the default byte order is big endian and not little endian (tested on Pixel2 and Samsung Galaxy S9)

This way I could decrypt old data and encrypt using the "safer" UTF-8encoding, which does not mess up byte order.

`public class Utf8Entity extends com.facebook.crypto.Entity {

private String mName;

Utf8Entity(String name) {
    super(name);
    this.mName = name;
}

@Override
public byte[] getBytes() {
    return this.mName.getBytes(StandardCharsets.UTF_8);
}

}`

`public class Utf16Entity extends com.facebook.crypto.Entity {

private static final byte[] byteArrayLE = new byte[]{-1, -2};

private String mName;

Utf16Entity(String name) {
    super(name);
    this.mName = name;
}

@Override
public byte[] getBytes() {

    byte[] bytes = this.mName.getBytes(StandardCharsets.UTF_16LE);
    ByteBuffer utf16LittleEndianBytes = ByteBuffer.allocate(bytes.length + byteArrayLE.length)
            // when String is encoded using UTF-16 without specifying the endianness, the byte array is prepended
            // with {-1,-2} bytes defining the endianness.
            .put(byteArrayLE)
            .put(bytes);
    return utf16LittleEndianBytes.array();
}

}`