TWRP not decrypting hammerhead device data using FDE

[brainchild0]

• [x] I am running an official build of TWRP, downloaded from https://twrp.me/Devices/
• [x] I am running the latest version of TWRP
• [x] I have read the FAQ (https://twrp.me/FAQ/)
• [x] I have searched for my issue and it does not already exist

Device codename: hammerhead TWRP version: 3.2.3-0 Android version: Resurrection Remix (based on LineageOS) adaptation of Android Oreo 8.1

WHAT STEPS WILL REPRODUCE THE PROBLEM?

Install Resurrection Remix and TWRP on a Nexus 5 device. Encrypt the device data.

WHAT IS THE EXPECTED RESULT?

TWRP should accept whatever password is set for the device when it prompts for a password on startup. It should then mount the data partitions to read and to modify device as instructed by user. If a non-password security measure is set, it should prompt appropriately (e.g. pattern grid for pattern-based security). If no password, pattern, or pin is set, then TWRP should start at the main menu with no security prompt.

WHAT HAPPENS INSTEAD?

TWRP cannot decrypt any data on my device. Since I encrypted my device, I have been unable to use TWRP for any system upgrades, backups, or other maintenance. I had no problems immediately before the device was encrypted.

Upon starting TWRP prompts for a password (i.e. character sequence, not pin or pattern), even though my chosen unlocking method is pattern. Even if I change the device security settings to allow me to boot the system without any password, pin, or pattern entry, TWRP still prompts for a password. The situation improves slightly if I enable, through the Android settings, the option to require unlocking at the time of boot. In this case, TWRP correctly identifies pattern unlocking as the chosen scheme, but does not accept the pattern I have chosen. The pattern always works correctly when given to the Android system, rather than to TWRP.

ADDITIONAL INFORMATION

I can navigate through the menus of TWRP if I cancel the password prompt, but I cannot use any features that depend on decrypting the data.

This behavior is the same using the official builds for all recent versions of TWRP, including 3.2.3-0, 3.2.2-0, and 3.2.1-0 (on hammerhead).

Possible duplicate of #1331 and #1310.

[harryharryharry]

I'm having a similar issue with kugo on omnirom 9 and TWRP 3.2.3-0, with encryption and secure start-up enabled: encryption with a pattern -> TWRP asks for password. encryption with a pin -> TWRP asks for a pattern.

Luckily encryption with a password does work correctly, but I'd much rather use a pattern.

[brainchild0]

I understand that TWRP is an open-source initiative, made possible by contributors who volunteer their personal time.

This issue appears to belong to a growing body of reports relating to devices running Lineage OS, or derivatives, and using encrypted storage, as evidenced by browsing the list of recently-tracked issues, some of which reference or are referenced by this report.

Since the number of related reports is expanding, and since most users filing these reports are explaining complete or near-complete inability to use TWRP on their devices, I am wondering whether the contributors would consider, if they have not already done, assigning a high priority to finding a workaround, or implementing a full resolution, to this problem, so that affected users can move forward soon.

Thank you.

[CaptainThrowback]

TWRP needs more contributors, otherwise there's no one to make the updates to the devices that are needed. @Dees-Troy did start working on an update for FDE decryption on Oreo+, but it still requires the device maintainers to update the blobs in their versions of TWRP, and there really aren't that many maintainers anymore. Many people have moved on to newer devices, and older devices get left by the wayside. That means new contributors are needed. So, your plea to contributors is nice, but the issue is that there aren't many contributors to work on the issue, even if a fix is added upstream for it.

[brainchild0]

I understand that testing on the target hardware is the only way to be certain that the software works as desired on that device, once it is developed and built, but from a development standpoint, is this support not largely a hardware-independent issue?

In the worst case, if a fix is confirmed to work for certain devices, can it simply be added to the builds for all devices, hoping that no problems occur for the end user?

Are you saying that certain features and fixes have been added upstream that have not found their way into the builds for all device targets?

I am confused about your comment about device maintainers updating their "blobs". I know the term in the generic sense but not how you are using it here.

[CaptainThrowback]

For working FDE decryption in TWRP, not only does the upstream code need to be updated, but each device needs specific blobs and services added to the device ramdisk for decryption on that specific device to work properly. These blobs, while similar for devices with the same SoC, are not identical for each device. So while the code updates are largely hardware-independent, the portion that isn't is what will prevent decryption from working on most devices.

Unfortunately there is no substitute for maintainers and testers when it comes to decryption support.

[brainchild0]

Thanks for the clarification. It seems that you are saying that the "blob" refers to a device-dependent build target that gets linked against the device-independent core when the release is built for each device? In any case, I understand now the need for the hardware-dependent features to be contributed by someone committed to maintenance for a particular device.

Currently, there is a great deal of confusion relating to users finding that decryption works unpredictably or not at all on their devices. They are asking for help on forums and creating tickets. I am not aware of any official statement or documentation indicating that the full feature set is not currently available for all devices, much less an actual feature matrix that shows support for particular devices. I have searched extensively on this topic, and have not found such information. Equally, up until @CaptainThrowback recently commented on my recent post, I was aware of no tickets on this system opened or commented on by members of the development team indicating that the missing functionality was a known issue, not an unknown bug. I suggest that any discrepancy between stated behavior and actual behavior should appear as a ticket on the system, even if for no other reason than that users can become aware of the issues already considered open and known to the developers after searching. Encrypting an Android device is irreversible, so it is very helpful that users know that some devices are not supported by TWRP, and which ones, before they make the decision to encrypt their device, with every expectation that TWRP will continue to work.

[CaptainThrowback]

https://twrp.me/faq/encryptionsupport.html

[brainchild0]

Thanks for the reference.

Notwithstanding the availability of the information to the contrary, I think that at the moment many users are under the impression that TWRP supports encryption, with little or no qualification. I agree that the FAQ entry adequately refutes this belief, but perhaps more users who need this information would actually see it if it were provided in a different place.

Perhaps a warning about encryption support could be placed on the download page, with a link to a matrix showing support for each device. Or perhaps device-specific information could be placed on the download page for each device. Users who are unaware that decryption must be implemented per device will tend to assume implicitly that if one user has success using encryption with TWRP, then all users will have the same success, without device type being an obstacle. It may seem obvious that this assumption is false to those familiar with the technical issues of the devices, but it is not obvious to everyone.

Another suggestion is to change the language of the FAQ entry slightly, such as:

Q: What is the status of encryption support in TWRP?
A: Support varies by device, and is not supported for some devices. To get information about support on your device, visit our status page for encryption support \<include link>.

The difference may seem minor, but the above form is much more noticeable to someone who is not already aware that support is device specific. Such a person might give little thought to the entry in its current form, because he has no reason to think that encryption is not supported on his device.

I think the reference you provided gives a clear explanation of the situation, and while my suggestions may seem excessively concerned with details, I think that it has become clear that the information is not getting propagated adequately to the user base, which creates difficulties both for users and developers.

I have no problem if this ticket gets closed, but I would encourage thinking about ways to make the communication more successful.

Thanks for your time.

[brainchild0]

As this issue is still open, I will simply suggest that, at an absolute minimum, please consider making a list of devices showing full, partial, or no support for encryption, to help device owners decide whether to migrate to encryption and to help prospective buyers select a device with required support. Thank you.

[CaptainThrowback]

And how would we source this list? Remember that TWRP depends not just on the main devs, but on device maintainers to provide such information. If you'd like to start a document that users can access and provide this information for their device, that would be helpful. Otherwise, we don't really have a starting point, besides the devices that have active maintainers. I think it's the no-longer-maintained devices that are the problem here.

[brainchild0]

I'm confused by your question. If the device maintainers provide information about the devices they support, then the source of the list would be a compilation of the information provided by each such maintainer. If some devices are no longer maintained, and support is limited to whatever has been provided by maintainers who are no longer contributing, then it is still helpful to have the available information shown about devices that are maintained, with the information for other devices, where appropriate, shown as unknown.

[CaptainThrowback]

The problem is that there are very few active device maintainers. Thus my question. The list would just be "unknown" for every device at this point.

[brainchild0]

Then I would suggest at least collecting this information for new submissions. Then, for actively maintained devices, the information will become available, and remain so in the future past the point that those devices are no longer maintained.

[rkoe]

Same issue here with TWRP 3.3.1-0 and Android 9 / Resurrection Remix 7. :(