TWRP 2.8.4.1-2.8.6.0 on Bacon doesn't decrypt /data

[jfeise]

The CM12 nightly from 0116 supports full-device encryption of /data. TWRP fails to decrypt that, saying 'wrong password'.

[skola28]

Also reporting this same issue. CM12 Nightly 1-16-2015, TWRP 2.8.4.1.
Using PIN unlock. Additionally, before I even set a pin password TWRP wasn't decrypting using "default_password" that Lollipop supposedly uses.

[Dees-Troy]

The CM 12 nightly for 2015-01-16 supports some kind of hardware encryption methods that aren't present in AOSP. Fortunately CM 12 source is open. Unfortunately, the process for decrypting in hardware isn't all that well documented. I pushed a couple of changes to my OnePlus One device files here: https://github.com/TeamWin/android_device_oneplus_bacon/tree/cm-11.0 The kernel I am using is here: https://github.com/TeamWin/android_kernel_oneplus_msm8974/tree/cm-12.0

I created this patch set to add some hardware decrypt methods to TWRP's decryption here: https://gerrit.omnirom.org/#/c/11542/

The process for using this at this point is somewhat manual. Build and boot TWRP using these repos and cherry pick the patch set. When prompted to decrypt, hit cancel. Go to the mount page and mount system and firmware. adb shell ln -sf /system/vendor /vendor and wait about 30 seconds then decrypt using the Decrypt button in the lower right. TWRP should report that it was decrypted successfully meaning that we've properly generated a decrypt key but for some reason the resulting dm-0 block device is not readable.

I'm not really sure where to take it from here. I repacked the boot image with some default.prop changes to enable full rooted debugging and got this log from a normal Android decrypt cycle: D/Cryptfs ( 204): crypt_ftr->fs_size = 25920798 I/Cryptfs ( 204): Using scrypt with keymaster for cryptfs KDF D/QSEECOMAPI: ( 204): QSEECom_get_handle sb_length = 0x2000 D/QSEECOMAPI: ( 204): App is already loaded QSEE and app id = 2 I/Cryptfs ( 204): Signing safely-padded object D/QSEECOMAPI: ( 204): QSEECom_dealloc_memory D/QSEECOMAPI: ( 204): QSEECom_shutdown_app, app_id = 2 D/ ( 204): HW based disk encryption is enabled D/ ( 204): HW based disk encryption is enabled D/ ( 204): Success loading QSEECom_create_key D/ ( 204): Success loading QSEECom_update_key_user_info D/ ( 204): Success loading QSEECom_wipe_key I/MQClient( 656): unable to connect to [/data/misc/location/mq/location-mq-s]:[No such file or directory] W/XTWiFiOS( 656): connection failure. count 6 D/Ulp_jni ( 656): JNI:In update_settings:currentContextType 0, currentGpsSetting 0, currentAgpsSetting 1,currentNetworkProvSetting 0,currentWifiSetting 0,currentBatteryCharging 0,currentEnhLocationServicesSetting 0 currentPipUserSetting 0 D/Ulp_jni ( 656): JNI:system_update. Event-4 E/QSEECOMAPI: ( 204): SUCESS::ioctl call to create encryption key for usage 1 success with ret = 0 D/ ( 204): HW based disk encryption is enabled I/Cryptfs ( 204): load_crypto_mapping_table: target_type = req-crypt I/Cryptfs ( 204): load_crypto_mapping_table: real_blk_name = /dev/block/platform/msm_sdcc.1/by-name/userdata, extra_params = fde_enabled I/Cryptfs ( 204): Password matches D/Cryptfs ( 204): test_mount_encrypted_fs(): Master key saved D/QSEECOMAPI: ( 204): QSEECom_get_handle sb_length = 0x2000 D/QSEECOMAPI: ( 204): App is already loaded QSEE and app id = 2 I/Cryptfs ( 204): keymaster version is 3 D/QSEECOMAPI: ( 204): QSEECom_dealloc_memory D/QSEECOMAPI: ( 204): QSEECom_shutdown_app, app_id = 2 E/VoldConnector( 656): NDC Command {10 cryptfs checkpw [scrubbed]} took too long (5946ms)

Hopefully someone with more time and fewer devices to maintain can take it a bit further and figure out what is still missing.

[jfeise]

Changing the encryption pwd fails even with cryptfs. That app is from the guy who wrote the book on Android security: http://www.nostarch.com/androidsecurity http://nelenkov.blogspot.com/2012/08/changing-androids-disk-encryption.html I hope he can provide some insights. His app works for my Nexus devices with CM12, just not for the OnePlus One.

[Dees-Troy]

I found that /system/bin/qseecomd is supposed to be running during the decrypt cycle. Running it before / during the decrypt cycle gives me a dmesg that's extremely similar to Android, but it still fails at the end. I added some extra logging to drivers/misc/qseecom.c in the kernel to try to help debug:

<3>[ 20.624936] QSEECOM: __qseecom_process_incomplete_cmd: fail:resp res= -68,app_id = 0,lstr = 12288 <3>[ 20.625003] QSEECOM: __qseecom_generate_and_save_key: process_incomplete_cmd FAILED, resp.result -68 <3>[ 20.625040] QSEECOM: __qseecom_generate_and_save_key: disable clk CLK_QSEE and ret -22 <3>[ 20.625104] QSEECOM: __qseecom_disable_clk: __qseecom_disable_clk <3>[ 20.625169] QSEECOM: qseecom_create_key: Failed to generate key on storage: -22 <3>[ 20.625205] QSEECOM: qseecom_ioctl: failed to create encryption key: -22 Unfotunately 2 of the big players during the decrypt cycle are the qseecomd and libQSEEComAPI.so which are both closed source proprietary pieces created by Qualcomm, so it makes it hard to debug and figure out what is wrong.

[JacqylFrost]

I'm having this issue on TWRP 2.8.3 with Lollipop, used pattern, enabled encryption, why did I do this to myself?

[jfeise]

No change with 2.8.5.1.

[abel408]

I am also on CM12 nightlies with TWRP 2.8.5.1. Cannot decrypt data from TWRP which makes the recovery pretty much useless.

[skola28]

Also reporting CM12 nightlies are not decrypting in TWRP 2.8.5.1. It is possible to install new nightlies via sideload however.

[skola28]

Suggest changing the title to specifically call out Lollipop as to not confuse this with the Cyanogenmod 11, and other hardware platform KitKat encryption issues that got resolved by 2.8.X

[Rashed97]

@Dees-Troy Have you tried something like this: https://github.com/CyanogenMod/android_device_oppo_msm8974-common/commit/a060186c93e5ed509bc3e2cd950efb981ade25b9

That should let QSEECOM run no matter what. Just an idea ;)

[Rashed97]

I believe you can still use adb sideload.

On Sun, Feb 22, 2015 at 10:11 AM, infinitnet notifications@github.com wrote:

I'm having the same problem with my M8 running Lollipop (ViperOneM8) and TWRP 2.8.5.0. I encrypted the device after setup and now tried to flash an OTA and discovered that I can't decrypt /data through TWRP (always says wrong password), which makes it impossible to install the very much needed update. Is there any fix for this in sight?

— Reply to this email directly or view it on GitHub https://github.com/TeamWin/Team-Win-Recovery-Project/issues/333#issuecomment-75439620 .

Sincerely,

Rashed Abdel-Tawab

[infinitnet]

Alright, it worked fine with adb sideload. However I needed the latest adb, which isn't available everywhere yet, so if anyone running Linux runs into the same issue (sideload error due to outdated adb), recent binaries can be taken from here: https://www.archlinux.org/packages/?name=android-tools

[aarmea]

I can't even seem to use the adb sideload workaround (disregarding the fact that it's a terrible idea to update without making a backup first). Instead, I get error: closed. Trying adb usb first doesn't help, either.

I'm running TWRP 2.8.5.1 with adb 1.0.31 on a Mac.

[jfeise]

I think that's a separate bug in 2.8.5.1. adb sideload works with 2.8.4.x. It may also have to do with your version of adb. 1.0.32 is the latest one. In 2.8.5.1, you can use adb push <cm-version> /cache, for example, and then install in recovery from there. That's what I currently do. But yes, not being able to do backups is a major issue.

[dbx4]

Another good alternative for flashing nightlies without /data access is using usb-otg. Here's what I do:

1. Copy ROM zip and Gapps zip onto USB stick.
2. Delete /data/dalvik-cache manually using file browser or terminal
3. Reboot into TWRP, wipe Cache partition
4. Insert USB stick, mount usb-otg, install ROM and then Gapps

[joshmule]

I do something similar:

1) Download ROM zip to USB 2) Reboot to TWRP with OTG cable inserted 3) Install ROM by selecting OTG storage and zipfile 4) Select Wipe Dalvik and Cache after the install, then reboot

This works as a workaround until encryption is truly supported in TWRP

[Diapolo]

I'm missing a comment I gave yesterday? Strange...

Using S4 i9506 (ks01lte) with TWRP 2.8.5.0 from here (http://forum.xda-developers.com/galaxy-s4/i9506-develop/recovery-twrp-2-8-1-0-touch-recovery-t2942159) and I'm also unable to decrypt my data partition. OS is official CM12 nightly (Android 5.0.2) and I also encrypted within that OS.

Is there any way I can help to track this issue down? I recently switched from CWM to TWRP in the hope I could use it with encrypted data :).

[theGrower]

How are people installing nightlies from /cache without decrypting /data? /data is where the updates are going. This fails for me every time. I have not tired and OTG, and I haven't update my ADB yet to try sideload again.

[urandom2]

I usually get some errors, but since the system is not encrypted, it will finish and update the system; assuming there are not systemic errors with the update that would also be seen with an unencrypted system

[jfeise]

You can install nightlies by sideloading them. Or copy them to /cache, then reboot into recovery and install from there. The install script tries to mount /data, which fails, but it continues installing the nightly. You can not do much else in recovery, in particular, you can not take backups of /data. That's really my main issue with the decryption failure.

[theGrower]

I’m realizing I’m having another very weird happening... When I’m booting up to CM12 I’m no longer asked for my encryption password. It has been like this since I first tried to decrypt in twrp after initial encryption. It just boots directly to the lock screen. Also, as far as installing from /cache is just not working. I’m not sure how anyone is getting this to work, so here is what I’m doing: First I moved the .zip into /cache using file manager; boot into recovery; click cancel at password (since it fails anyway); Click Install; move to /cache and choose .zip; swipe to Confirm flash. At this time TWRP temporarily shows the running script, then reboots back into recovery. If I’m doing something wrong, I’d love to be set straight!

[joshmule]

Yes, /data isn't needed at all for ROM upgrades. Put the zipfile on the USB stick and connect via OTG cable. Yes /data mount will fail, but unneeded for upgrade.

[ApolloLV]

I always update like this: Download the new nightly with the integrated CM update manager. run cp /storage/sdcard0/cmupdater/.zip /cache/nightly.zip in a root shell inside a terminal emulator, reboot into recovery, cancel decryption. Install zip from /cache (which is not encrypted), reboot into system.

[mjb2000]

These work around work fine for updating CM with an inaccessible /data - but does anyone know if there is a timeline or plan to get access to an encrypted /data within TWRP?

[Diapolo]

@mjb2000 Yeah I'm also interested in a fix and a timeline, because TWRP just get's a perfect CWM replacement if accessing encrypted /data is working :).

[skola28]

The last I heard was TWRP will update when there is a stable rom version available.

[theGrower]

Cannot sideload. "error: insufficient permissions for device" Cannot install from /cache so please stop repeating yourselves. Perhaps you can, and congratulations on your success. All that happens when I try is TWRP reboots itself.

Even weirder - I'm still not getting prompted to put in my encryption password at boot up which is quite concerning. TWRP pretty much fails at everything right now. Cannot decrypt, cannot mount anything.

Probably going to need to format everything and start anew. If TWRP and CM12 nightlies are not going to be basically functional, I guess it is back to stock CM11s ROM. To which I say, BOOOOOOOO!!

EDIT/UPDATE After some investigation and re-installing ADB and fastboot a few times on my linux box, I was able to sideload the newest nightly, or so it seemed. It appeared to work, but when rebooting into CM12 and checking the version it was showing the old nightly version. So I attempted again to install from /cache and again it appeared to work. Check the version and it indeed did update. I don't understand why it worked this time, but I'm not arguing with it. However, still remained absolutely perplexed as to why I am no longer prompted for my decryption password at boot. It acts as if it is not encrypted; however, TWRP still cannot decrypt.

[ashish2sharma]

@theGrower You can check by changing the lock type from settings.. It will show option to ask for password on boot or not to ask. You might have selected the other option.

[theGrower]

@ashish2sharma I did. This option was not on. I actually got sick of the issues and bailed completely on the nighties. Too much trouble currently without having all the functionality I desire. I'll shoot for a Snapshot when they are available. I'm back to stock rooted ROM. Not encrypted for the time being as that breaks the compass and navigation gets silly.

[jfeise]

Still no change with 2.8.6.0.

[tomstoms]

I have same problem, with CM12 28.03. nightly and TWRP 2851.

so is there any ETA for fixing this?

[troublekick]

Twrp has become a pain to use with this decryption issue! Waiting for an ETA too.. ,

[gro-gg]

I've had the same problem. Now I am using CyanDelta Updater to install new nightlies. This works with TWRP 2.8.5.2 (and maybe wtih older versions).

[marvinandro]

adb push has been my way around this but can't daily run to a PC to do the upgrades. ETA would be awesome.

[ericfowler303]

Another recent change to think about for those using larger pattern unlocks: http://review.cyanogenmod.org/#/c/93335/ http://review.cyanogenmod.org/#/c/93353/

[denysvitali]

Same here guys... following.

[Kri22]

Decryption on OxygenOS isn't possible either. It doesn't seem to be a specific problem on CyanogenMod only...

[denysvitali]

It should be something related to the new Lollipop encryption system. Maybe looking at AOSP source code will give some infos about the problem...

[theGrower]

Is this a problem for TWRP on any other device running CM12 nightlies? If not, then it sounds like a OPO specific problem... Does anyone know if other devices have this issue?

EDIT: *Did the smart thing and searched for other phones with this issues. Looks like at least the Note3 has this issue as well. I found the HTC one had the issue as well, but on a KitKat build. ... Perhaps it is with the ASOP? I'm no programmer, just a hack that nearly bricks his phone and tablet monthly, so I have nothing truly insightful to offer, FYI. O_o

[corrupt]

It should be a problem on all devices. I haven't heard of a single case on xda in which encryption in TWRP has worked running any version of Lollipop. I'm running CM12 on amami and it certainly doesn't.

[urandom2]

I have an OPO and can confirm the decryption issue there, but my 2012 nexus 7 has cm12 with no decryption issues.

edit: for clarity, my nexus 7 is a first generation

[Diapolo]

I can confirm this is also not working on ks01lte and I consider the encryption/decryption support of /data completely broken!

[cha0sify]

I have also a Nexus 7 (2Gen LTE (deb)) with cm12, and decryption dos NOT work with twrp... So it is not OPO Specific..

[Zylvo]

Yeah, hlte (note 3) on cm12.1(android 5.1) cannot decrypt /data too :< Hope devs will figure out what's wrong

[jfeise]

Decryption works with the latest TWRP and CM12 on Nexus 4 and the 1st gen Nexus 7 (grouper.) It is an issue with OPO (and possibly other devices) due to where the key is stored. The OPO stores it in hardware, and there is some proprietary library that retrieves it. Accessing that from TWRP is the problem.

[theGrower]

Probably a dumb question - I've done a lot of searching for alternate disk encryption rather than using Android's default encryption and can find nothing. I assume this is impossible, improbable, or just too problematic to be able to do? I assume it happens to be in the way the bootloader handles decryption at boot? I have many different options to encrypt on my Linux box...

[urandom2]

First, the bootloader does not handle encryption. Second, decryption is a difficult process because there are fewer input devices on Android than your Linux box. Android currently gets around this by building a decryption routine into the system and then not encrypting the system [only the userspace is encrypted]. Linux does disk encryption at many levels, but for full disk encrypiton, employs an interactive booloader that Android does not have. This lack of interactivity caused issues for researchers working on ARMORED: https://www1.informatik.uni-erlangen.de/tresor .

I think that android encryption is good enough in its current state, but most people would like FDE but it is difficult on a phone.

Let me know if I have something horribly wrong, as I have not developed or read through most of the code I am discussing.

[theGrower]

I've been thinking it may just make more sense for my purposes to keep any sensitive files in an encrypted folder on my devices as opposed to encrypting the entire user space. I don't generally keep much that is sensitive enough to worry about. My biggest worry is my password keeper, but that is held in its own encrypted file as well. Additionally, I don't need full encryption for my job as I have a work phone that is encrypted and set up by our IT department. For piece of mind, I'd like the encryption to just work, but perhaps my suggestion above would suffice until TWRP and CM12 begin to play nice in the sandbox.

[urandom2]

Know that cm-12 has support for encrypted containers: http://www.cyanogenmod.org/blog/all-about-l-part-1

p.s. they call it secure storage

[hifi1231]

Hello, I have question to you guys. As I have encrypted 1+1 with TWRP and can't acces to that TWRP cuz of "password" (even if there is none password to the phone lol), how can I instal AK Kernel and xposed right now? As I understand it's possible to connect OTG pendrive and I will se .zip files on that pendrive, buy will it flash to system when it's encrypted?