search

ADeadTrousers / twrp_device_Unihertz_Atom_LXL

Common device tree for the Unihertz Atom L and XL
Apache License 2.0
7 stars 3 forks source link

Decryption (still) isn't working #6

Open ADeadTrousers opened 2 years ago

ADeadTrousers commented 2 years ago

First boot messages look promising

[    0.774271] .(6)[1:swapper/0]TrustKernel TEE Driver initialization
[    0.774851] .(6)[1:swapper/0]TEE core: Alloc the misc device "tkcoredrv" (id=0)
[    0.774913] .(6)[1:swapper/0]tkcoreos-rev: 0.10.5-gp
[    0.774986] .(6)[1:swapper/0]tkcoredrv: tee_log_irq id = 15
[    0.775144] .(5)[123:tee-log][    0.000000]: <0>INF TKCore:init_teecore:71: teecore: init app mem
[    0.775152] .(6)[1:swapper/0]TKCore misc: Register the misc device "tkcoredrv" (id=0,minor=125)
[    0.775160] .(5)[123:tee-log][    0.000000]: <0>INF TKCore:init_teecore:74: teecore: init ctors
[    0.775166] .(5)[123:tee-log][    0.000000]: <0>INF TKCore:init_teecore:78: teecore: init time source
[    0.775173] .(5)[123:tee-log][    0.000000]: <0>INF TKCore:time_source_init:94: Frequency mult = 1 divisor = 13
[    0.775179] .(5)[123:tee-log][   10.502793]: <0>INF TKCore:init_teecore:82: teecore: init calls
[    0.775186] .(5)[123:tee-log][   10.503770]: <0>INF TKCore:spi_init:923: Setup SPI0 @000000001100a000
[    0.775192] .(5)[123:tee-log][   10.504602]: <0>INF TKCore:init_teecore:85: teecore: done
[    0.775198] .(5)[123:tee-log][   10.505296]: <0>INF TKCore:license_early_init:1934: TEE: init license
[    0.775204] .(5)[123:tee-log][   10.506549]: <0>INF TKCore:parse_cert:546: projectid: 4561
[    0.775210] .(5)[123:tee-log][   10.507268]: <0>INF TKCore:dump_license:1926: Project: [Unihertz Atom_XL mt6771]
[    0.775216] .(5)[123:tee-log][   10.508221]: <0>INF TKCore:set_pl_key_id:397: TURKEY ID = 1
[    0.775222] .(5)[123:tee-log][   10.508938]: <0>MSG TKCore:dump_tee_params:262: VERSION:    0x30000
[    0.775228] .(5)[123:tee-log][   10.509744]: <0>MSG TKCore:dump_tee_params:265: UART_BASE:  0x11002000
[    0.775234] .(5)[123:tee-log][   10.510583]: <0>MSG TKCore:dump_tee_params:268: RPMB KEY:   Y
[    0.775240] .(5)[123:tee-log][   10.511323]: <0>MSG TKCore:dump_tee_params:271: RPMB SIZE:  0x1000000
[    0.775246] .(5)[123:tee-log][   10.512152]: <0>MSG TKCore:dump_tee_params:274: SOC ID:     Y
[    0.775252] .(5)[123:tee-log][   10.512890]: <0>MSG TKCore:dump_tee_params:276: RTC TIME:   1656076992
[    0.775258] .(5)[123:tee-log][   10.513730]: <0>INF TKCore:init_primary_helper:211: Primary cpu booted

But as soon as it starts to load the certs shit hits the fan

[    6.402005] .(2)[123:tee-log][   25.133099]: <7>DBG TKCore:invoke_command:417: cmd_id=2820 nParamTypes=0x2
[    6.402012] .(2)[123:tee-log][   25.134029]: <7>ERR TKCore:invoke_command:723: Unsupported command id: 0xb04
[    6.402018] .(2)[123:tee-log][   25.134949]: <7>DBG TKCore:tee_ta_invoke_command:1891:   => Error: ffff0006 of 4
[    6.402024] .(2)[123:tee-log][   25.137160]: <1>DBG TKCore:invoke_command:417: cmd_id=0 nParamTypes=0x555
[    6.402030] .(2)[123:tee-log][   25.138151]: <1>ERR TKCore:tee_rpmb_get_dev_info:1014: send rpmb command failed with 0xffff0009
[    6.402036] .(2)[123:tee-log][   25.139283]: <1>ERR TKCore:tee_rpmb_init:1131: Failed to retrieve rpmb device info with 0xffff0009
[    6.402042] .(2)[123:tee-log][   25.140447]: <1>ERR TKCore:init_truststore:132: Init rpmb device failed with 0xffff0009
[    6.402048] .(2)[123:tee-log][   25.141489]: <1>MSG TKCore:truststore_source_init_with_type:122: Init truststore with type=4
[    6.402054] .(2)[123:tee-log][   25.142634]: <1>INF TKCore:init_truststore:274: DEFAULT TRUSTSTORE Setup ... Done
[    6.402060] .(2)[123:tee-log][   25.143628]: <1>INF TKCore:license_init:2026: Check secondary external certificate
[    6.402065] .(2)[123:tee-log][   25.147475]: <0>ERR TKCore:extract_secondary_cert:822: Invalid block size: 0
[    6.402071] .(2)[123:tee-log][   25.152118]: <4>ERR TKCore:init_secondary_cert:922: Failed to extract secondary cert with 0xffff0010
[    6.402077] .(2)[123:tee-log][   25.153361]: <5>INF TKCore:license_init:2034: Check full external certificate
[    6.402083] .(2)[123:tee-log][   25.158329]: <6>DBG TKCore:wq_rpc:56: [00000000: 1] sleep <thread_big_lock>
[    6.402088] .(2)[123:tee-log][   25.159263]: <1>DBG TKCore:wq_rpc:56: [00000000: 2] sleep <thread_big_lock>
[    6.402094] .(2)[123:tee-log][   25.163512]: <7>ERR TKCore:extract_cert:943: Invalid veriblk size: 0
[    6.402099] .(2)[123:tee-log][   25.164338]: <7>ERR TKCore:init_external_cert:1283: Failed to extract cert: 0xffff0010
[    6.402105] .(2)[123:tee-log][   25.165424]: <7>ERR TKCore:license_init:2036: Invalid external cert: 0xffff0010
[    6.402111] .(2)[123:tee-log][   25.179937]: <4>INF TKCore:license_expiring:1112: expiring counter: 838
[    6.402116] .(2)[123:tee-log][   25.180800]: <4>INF TKCore:license_init:2061: VERIFY_STATE: 1 TRIAL_STATE: 2
[    6.402122] .(2)[123:tee-log][   25.181750]: <4>DBG TKCore:wq_rpc:56: [00000000: 1] wake <thread_big_lock>
[    6.402127] .(2)[123:tee-log][   25.182752]: <4>DBG TKCore:wq_rpc:56: [00000000: 0] sleep <thread_big_lock>
[    6.402133] .(2)[123:tee-log][   25.183664]: <6>ERR TKCore:tee_ta_rpc_load:1347: load TA failed with 0xffff0007
[    6.402140] .(2)[123:tee-log][   25.184622]: <6>DBG TKCore:tee_dispatch_open_session:144: TA <b46325e6-5c90-8252-2eada8e32e5180d6> => Error: ffff0007<...>
[    6.402144] .(2)[123:tee-log] of 2
[    6.402150] .(2)[123:tee-log][   25.186086]: <6>DBG TKCore:wq_rpc:56: [00000000: 2] wake <thread_big_lock>
[    6.402155] .(2)[123:tee-log][   25.187211]: <1>ERR TKCore:tee_ta_rpc_load:1347: load TA failed with 0xffff0007
[    6.402162] .(2)[123:tee-log][   25.188169]: <1>DBG TKCore:tee_dispatch_open_session:144: TA <9ef77781-7bd5-4e39-965f20f6f211f46b> => Error: ffff0007<...>
[    6.402166] .(2)[123:tee-log] of 2
[    6.402171] .(2)[123:tee-log][   25.189627]: <1>DBG TKCore:wq_rpc:56: [00000000: 0] wake <thread_big_lock>

The trustkernel intialization is taken from stock rom https://github.com/ADeadTrousers/twrp_device_Unihertz_Atom_LXL/blob/twrp-11.0-nocrypt/recovery/root/vendor/etc/init/trustkernel.rc#L57

In comparison to the old stock rom a few more parameters need to be set https://github.com/ADeadTrousers/twrp_device_Unihertz_Atom_LXL/blob/twrp-10.0/recovery/root/vendor/etc/init/trustkernel.rc#L54

The most notable one is --rpmbdev which is mentioned in the error messages. But it doesn't help to leave them out. So my guess is that teed NEEDS those. The same goes with the second --prot.

So the whole problem is: teed is not able to load the certs android.hardware.gatekeeper@1.0-service is not able to authenticate itself against teed android.hardware.keymaster@4.0-service.trustkernel is not able to authenticate itself against teed vold is not able to utilize gatekeeper and keymaster to decrypt TWRP isn't booting, the screen goes blank and after a few seconds the phone reboots.

Deactivating TW_INCLUDE_CRYPTO lets TWRP boot but without decryption.

ADeadTrousers commented 2 years ago

/dev/rpmb0 exists while /dev/mmcblk0rpmb does not. But removing either or both doesn't change a bit in the logs.

I also tried to copy the --prot folders /mnt/vendor/persist/t6 and /mnt/vendor/protect_f/tee using -p to even mimicking the timestamps but it didn't help either. https://github.com/ADeadTrousers/twrp_device_Unihertz_Atom_LXL/commit/0992a63ea338c6356b8f8e4512981df018963b88#diff-e826647ea3da046036fd2fca5b6933d10545a79a77d8361c529881e210b8e845 (This step is required or otherwise the recovery WILL overwrite the files in those folders and the system won't be able to decrypt anymore)

Changing the group of every part that is evolved did do shit also https://github.com/ADeadTrousers/twrp_device_Unihertz_Atom_LXL/commit/0992a63ea338c6356b8f8e4512981df018963b88

lopestom commented 2 years ago

Hi AD; One question: your device is update to A11?

looking for anything about:

on post-fs
    # Set /mnt/vendor/persist/rpmb
    chown system system /mnt/vendor/persist
    chmod 0771 /mnt/vendor/persist
    restorecon_recursive /mnt/vendor/persist

    mkdir /mnt/vendor/persist/rpmb
    chown system system /mnt/vendor/persist/rpmb
    chmod 0771 /mnt/vendor/persist/rpmb
    restorecon_recursive /mnt/vendor/persist/rpmb

    # Set other rpmb
    mkdir /persist/rpmb
    mkdir /vendor/persist/rpmb
    chmod 0771 /persist/rpmb
    chmod 0771 /vendor/persist/rpmb
    chown system system /persist/rpmb
    chown system system /vendor/persist/rpmb
    restorecon_recursive /persist
    restorecon_recursive /vendor/persist

or

on early-fs
    start vold

read the \system\system_ext\etc\init\hw\vendor_init_as_system.rc file you can understand/translate to us.

ADeadTrousers commented 2 years ago

hi.

Yes I'm on A11 (matter of factly). Sadly I don't have these rpmb settings on my vendor_init_as_system.rc. It's just for creating some directories in various parts of the vendor portion anyway. But as long as there are no services or other parts of the rom referring to them they are useless. Like on mine the directories for teed are created here https://github.com/ADeadTrousers/twrp_device_Unihertz_Atom_LXL/blob/twrp-11.0-nocrypt/recovery/root/init.recovery.trustkernel.rc and used here: https://github.com/ADeadTrousers/twrp_device_Unihertz_Atom_LXL/blob/twrp-11.0-nocrypt/recovery/root/vendor/etc/init/trustkernel.rc

ADeadTrousers commented 2 years ago

tee_check_keybox is run after teed finishes with property:vendor.trustkernel.ready=true Your implementation is slightly different to mine (teei_daemon instead of teed) It's with recovery so no A/B.

lopestom commented 2 years ago

tee_check_keybox is run after teed finishes with property:vendor.trustkernel.ready=true Your implementation is slightly different to mine (teei_daemon instead of teed) It's with recovery so no A/B.

I have 2+ devices with A9 &A10........ 3 devices with A11 and one have same trustkerne that you have..... I want the decrypt as you want with this trusttkernel option and I only know that you solved. Never other guy solved that! Look my DT (thats a not full used to create img file) as referencied.

I read about some case that need put something for \data can "working" or "read" before/after mount points. So because that maybe need put 

on early-fs
    start vold

Looking for post-fs-data like this: only a reference

ADeadTrousers commented 2 years ago

I think your file layout is a little bit odd. You should use the same structure as in the stock rom. So teed should be situated under /vendor/bin and not /system/bin. Use this as a guideline on which file to be placed where https://github.com/ADeadTrousers/twrp_device_Unihertz_Atom_LXL/blob/twrp-11.0-nocrypt/proprietary-files.txt Next I would recommend you REMOVE the encryption for now: https://github.com/ADeadTrousers/twrp_device_Unihertz_Atom_LXL/blob/twrp-11.0-nocrypt/BoardConfigCommon.mk#L106 That way the recovery should start without any troubles and you can check if the vendor parts needed for the encryption are working. Simply by connecting via adb shell and running the programs teed, keymaster and gatekeeper with their respective parameters taken from the init.rc-scripts. That way you can figure out which parts might be missing or not working.

ADeadTrousers commented 2 years ago

The logs I mentioned in the first post also happen under LOS and the decryption works fine there. So my next best guess is that selinux might be the culprit or rather the lack of rules for vold to boot up correctly.

The black screen rebooting felt the same as under LOS while debugging an error in the rules. The phone simply reboots or boots into the bootloader. So I thought maybe setting selinux to permissive would at least let me boot with decryption enabled. But this seems to be impossible. When setting the kernel command line to androidboot.selinux=permissive the phone simply boots into the bootloader although I told it to boot into the recovery. Trying to do setenforce permissive or similar on the android shell returns a rather odd error setenforce: Couldn't set enforcing status to 'permissive': Invalid argument although I checked it several times.

ADeadTrousers commented 2 years ago

So either unihertz or mediatek "disabled" the switching of the selinux mode in the kernel (for their trustkernel environment?) OR this is a new "security" feature of android all along.

ADeadTrousers commented 1 year ago

I was able to patch up the kernel so that selinux is running in "permissive" mode. And still the kernel won't boot with crypto enabled.

So I decided to start from scratch and remove all vendor files. Now the kernel is booting with crypto enabled??? WTF??? Okay, so it seems like that any teensy tiny bit of teed just present in the recovery will have the kernel crashing. Namely: vendor/app/t6/*, vendor/bin/teed, vendor/bin/tee_check_keybox and vendor/build.prop But those are the files needed for decrypt to work.

ADeadTrousers commented 1 year ago

Placing teed somewhere else besides vendor/bin doesn't help either.

ADeadTrousers commented 1 year ago

Even renaming the file to something completely different didn't help either, How in the world is the kernel still able to detect this file and trigger this chaos? It's not called from any program or startup script.

ADeadTrousers commented 1 year ago

Aaarghh... Although the recovery.img is still below the size of the partition the included ramdisk seems to be too big. As soon as it is bigger than 16.8 MB compressed or 52 MB uncompressed the device won't boot.

This took me way too long to figure out than I'm comfortable to admit.

So now I can REALLY start to work on the decryption.

12-15 21:43:17.280   352   352 I recovery: fscrypt_initialize_systemwide_keys
12-15 21:43:17.280   352   352 W recovery: [libfs_mgr]Warning: unknown flag: resize
12-15 21:43:17.281   352   352 I recovery: Key exists, using: /data/unencrypted/key
12-15 21:43:17.285   313   313 I hwservicemanager: getTransport: Cannot find entry android.hardware.keymaster@3.0::IKeymasterDevice/default in either framework or device manifest.
12-15 21:43:17.285   352   352 I recovery: List of Keymaster HALs found:
12-15 21:43:17.286   352   352 I recovery: Keymaster HAL #1: HardwareKeymasterDevice from TrustKernel SecurityLevel: TRUSTED_ENVIRONMENT HAL: android.hardware.keymaster@4.1::IKeymasterDevice/default
12-15 21:43:17.286   351   351 D KeymasterHAL: TrustKernelKeymasterImplementation.cpp:213: Invoke cmdId 27 at vendor/mediatek/proprietary/trustzone/trustkernel/source/services/keymaster/TrustKernelKeymasterImplementation.cpp:268
12-15 21:43:17.289   351   351 D KeymasterHAL: TrustKernelKeymasterImplementation.cpp:213: Invoke cmdId 28 at vendor/mediatek/proprietary/trustzone/trustkernel/source/services/keymaster/TrustKernelKeymasterImplementation.cpp:327
12-15 21:43:17.292   352   352 I recovery: Using HardwareKeymasterDevice from TrustKernel for encryption.  Security level: TRUSTED_ENVIRONMENT, HAL: android.hardware.keymaster@4.1::IKeymasterDevice/default
12-15 21:43:17.292   351   351 D KeymasterHAL: TrustKernelKeymasterImplementation.cpp:213: Invoke cmdId 16 at vendor/mediatek/proprietary/trustzone/trustkernel/source/services/keymaster/TrustKernelKeymasterImplementation.cpp:1473
12-15 21:43:17.295   351   351 E KeymasterHAL: TrustKernelKeymasterImplementation.cpp:1476: TEE return -62
12-15 21:43:17.295   352   352 E recovery: begin failed, code -62
12-15 21:43:17.295   351   351 D KeymasterHAL: TrustKernelKeymasterImplementation.cpp:213: Invoke cmdId 26 at vendor/mediatek/proprietary/trustzone/trustkernel/source/services/keymaster/TrustKernelKeymasterImplementation.cpp:1884
12-15 21:43:17.302   352   352 I recovery: Key upgraded: /data/unencrypted/key
12-15 21:43:17.302   351   351 D KeymasterHAL: TrustKernelKeymasterImplementation.cpp:213: Invoke cmdId 16 at vendor/mediatek/proprietary/trustzone/trustkernel/source/services/keymaster/TrustKernelKeymasterImplementation.cpp:1473
12-15 21:43:17.305   351   351 D KeymasterHAL: TrustKernelKeymasterImplementation.cpp:213: Invoke cmdId 17 at vendor/mediatek/proprietary/trustzone/trustkernel/source/services/keymaster/TrustKernelKeymasterImplementation.cpp:1557
12-15 21:43:17.308   351   351 D KeymasterHAL: TrustKernelKeymasterImplementation.cpp:1563: Update output 64B
12-15 21:43:17.308   351   351 D KeymasterHAL: TrustKernelKeymasterImplementation.cpp:213: Invoke cmdId 17 at vendor/mediatek/proprietary/trustzone/trustkernel/source/services/keymaster/TrustKernelKeymasterImplementation.cpp:1557
12-15 21:43:17.311   351   351 D KeymasterHAL: TrustKernelKeymasterImplementation.cpp:1563: Update output 0B
12-15 21:43:17.311   351   351 D KeymasterHAL: TrustKernelKeymasterImplementation.cpp:213: Invoke cmdId 18 at vendor/mediatek/proprietary/trustzone/trustkernel/source/services/keymaster/TrustKernelKeymasterImplementation.cpp:1663
12-15 21:43:17.314   352   352 I recovery: Detected support for FS_IOC_ADD_ENCRYPTION_KEY
12-15 21:43:17.316   352   352 I recovery: Installed fscrypt key with ref e71628b87d6c6bb2 to /data
12-15 21:43:17.316   352   352 I recovery: Added fscrypt-provisioning key for e71628b87d6c6bb2 to session keyring
12-15 21:43:17.323   352   352 I recovery: Wrote system DE key reference to:/data/unencrypted/ref
12-15 21:43:17.323   352   352 I recovery: Installed fscrypt key with ref e0e667de4d6fd874 to /data
12-15 21:43:17.323   352   352 I recovery: Added fscrypt-provisioning key for e0e667de4d6fd874 to session keyring
12-15 21:43:17.326   352   352 I recovery: Wrote per boot key reference to:/data/unencrypted/per_boot_ref
12-15 21:43:17.326   352   352 I recovery: fscrypt_init_user0
12-15 21:43:17.326   352   352 I recovery: Preparing: /data/misc/vold/user_keys
12-15 21:43:17.331   352   352 I recovery: Preparing: /data/misc/vold/user_keys/ce
12-15 21:43:17.331   352   352 I recovery: Preparing: /data/misc/vold/user_keys/de
12-15 21:43:17.331   352   352 I recovery: fscrypt::load_all_de_keys
12-15 21:43:17.333   352   352 E recovery: Version mismatch, expected 1 got \3B

Although fscrypt_initialize_systemwide_keys seems to be getting keys for decryption they don't seem to be correct as fscrypt_init_user0 fails with a version mismatch because it reads \3B whereas it should find a number at least. This means the decryption didn't work.

The error code -62 means the key need to be upgraded as the recovery runs with PLATFORM_SECURITY_PATCH := 2099-12-05 whereas system currently runs with PLATFORM_SECURITY_PATCH := 2022-12-05. So maybe something with the updgrade procedure is producing wrong keys. I'd need to tinker a bit with this.

Oh, and I neither know why the recovery is querying for 3.0::IKeymasterDevice nor why the vendor service suddenly becoming 4.1::IKeymasterDevice when it was 4.0::IKeymasterDevice during the HAL registration. But it (kinda) works. So I got this going for me which is nice.

ADeadTrousers commented 1 year ago

Leaving out the CMD_UPGRADE_KEY by using the same PLATFORM_SECURITY_PATCH as the system it still isn't working:

12-19 19:53:23.270   352   352 I recovery: fscrypt_initialize_systemwide_keys
12-19 19:53:23.271   352   352 W recovery: [libfs_mgr]Warning: unknown flag: resize
12-19 19:53:23.271   352   352 I recovery: Key exists, using: /data/unencrypted/key
12-19 19:53:23.274   352   352 E cutils-trace: Error opening trace file: No such file or directory (2)
12-19 19:53:23.275   313   313 I hwservicemanager: getTransport: Cannot find entry android.hardware.keymaster@3.0::IKeymasterDevice/default in either framework or device manifest.
12-19 19:53:23.276   352   352 I recovery: List of Keymaster HALs found:
12-19 19:53:23.276   352   352 I recovery: Keymaster HAL #1: HardwareKeymasterDevice from TrustKernel SecurityLevel: TRUSTED_ENVIRONMENT HAL: android.hardware.keymaster@4.1::IKeymasterDevice/default
12-19 19:53:23.276   351   351 D KeymasterHAL: TrustKernelKeymasterImplementation.cpp:213: Invoke cmdId 27 at vendor/mediatek/proprietary/trustzone/trustkernel/source/services/keymaster/TrustKernelKeymasterImplementation.cpp:268
12-19 19:53:23.279   351   351 D KeymasterHAL: TrustKernelKeymasterImplementation.cpp:213: Invoke cmdId 28 at vendor/mediatek/proprietary/trustzone/trustkernel/source/services/keymaster/TrustKernelKeymasterImplementation.cpp:327
12-19 19:53:23.282   352   352 I recovery: Using HardwareKeymasterDevice from TrustKernel for encryption.  Security level: TRUSTED_ENVIRONMENT, HAL: android.hardware.keymaster@4.1::IKeymasterDevice/default
12-19 19:53:23.282   351   351 D KeymasterHAL: TrustKernelKeymasterImplementation.cpp:213: Invoke cmdId 16 at vendor/mediatek/proprietary/trustzone/trustkernel/source/services/keymaster/TrustKernelKeymasterImplementation.cpp:1473
12-19 19:53:23.286   351   351 D KeymasterHAL: TrustKernelKeymasterImplementation.cpp:213: Invoke cmdId 17 at vendor/mediatek/proprietary/trustzone/trustkernel/source/services/keymaster/TrustKernelKeymasterImplementation.cpp:1557
12-19 19:53:23.288   351   351 D KeymasterHAL: TrustKernelKeymasterImplementation.cpp:1563: Update output 64B
12-19 19:53:23.289   351   351 D KeymasterHAL: TrustKernelKeymasterImplementation.cpp:213: Invoke cmdId 17 at vendor/mediatek/proprietary/trustzone/trustkernel/source/services/keymaster/TrustKernelKeymasterImplementation.cpp:1557
12-19 19:53:23.292   351   351 D KeymasterHAL: TrustKernelKeymasterImplementation.cpp:1563: Update output 0B
12-19 19:53:23.292   351   351 D KeymasterHAL: TrustKernelKeymasterImplementation.cpp:213: Invoke cmdId 18 at vendor/mediatek/proprietary/trustzone/trustkernel/source/services/keymaster/TrustKernelKeymasterImplementation.cpp:1663
12-19 19:53:23.295   352   352 I recovery: Detected support for FS_IOC_ADD_ENCRYPTION_KEY
12-19 19:53:23.296   352   352 I recovery: Installed fscrypt key with ref e71628b87d6c6bb2 to /data
12-19 19:53:23.296   352   352 I recovery: Added fscrypt-provisioning key for e71628b87d6c6bb2 to session keyring
12-19 19:53:23.302   352   352 I recovery: Wrote system DE key reference to:/data/unencrypted/ref
12-19 19:53:23.303   352   352 I recovery: Installed fscrypt key with ref 903f5c39d49e8da2 to /data
12-19 19:53:23.303   352   352 I recovery: Added fscrypt-provisioning key for 903f5c39d49e8da2 to session keyring
12-19 19:53:23.305   352   352 I recovery: Wrote per boot key reference to:/data/unencrypted/per_boot_ref
12-19 19:53:23.305   352   352 I recovery: fscrypt_init_user0
12-19 19:53:23.305   352   352 I recovery: Preparing: /data/misc/vold/user_keys
12-19 19:53:23.309   352   352 I recovery: Preparing: /data/misc/vold/user_keys/ce
12-19 19:53:23.309   352   352 I recovery: Preparing: /data/misc/vold/user_keys/de
12-19 19:53:23.310   352   352 I recovery: fscrypt::load_all_de_keys
12-19 19:53:23.311   352   352 E recovery: Version mismatch, expected 1 got \B3

Taken from a particular source file which matches the mentioned line numbers from the log I got these command ids:

#define CMD_GENERATE_KEY 13
#define CMD_GET_KEY_CHARACTER_SIZE 14
#define CMD_GET_KEY_CHARACTER 15
#define CMD_BEGIN 16
#define CMD_UPDATE 17
#define CMD_FINISH 18
#define CMD_IMPORT_KEY 19
#define CMD_EXPORT_KEY 20
#define CMD_DELETE_KEY 21
#define CMD_DELETE_ALL_KEY 22
#define CMD_ABORT 23
#define CMD_ATTEST_KEY 24
#define CMD_CONFIGURE 25
#define CMD_UPGRADE_KEY 26
#define CMD_GET_HMAC_SHARING_PARAM 27
#define CMD_COMPUTE_SHARED_HMAC 28
#define CMD_VERIFY_AUTHORIZATION 29

So the second call of CMD_UPDATE returns with an empty result ("Update output 0B") The first call always returns the same key e71628b87d6c6bb2 and the second differs on every try I've seen so far.

ADeadTrousers commented 1 year ago

If I'm not mistaken in bootable/recovery/crypto/fscrypt/Keymaster.cpp the method updateCompletely is run repeatedly until it returns no more data. Therefore the second call with Update output 0B is correct.

ADeadTrousers commented 1 year ago

I've also checked all the involved binaries for properties that are being read: teed:

ro.product.brand
ro.product.model
ro.board.platform

keymaster:

ro.boot.verifiedbootstate
ro.boot.vbmeta.digest
ro.build.version.release
ro.build.version.security_patch

Comparing the values between twrp and system they are identical so this also cannot be the problem.

ADeadTrousers commented 1 year ago

fstab has the following flags for userdata

/dev/block/by-name/userdata /data ext4 noatime,nosuid,nodev,noauto_da_alloc,errors=panic,inlinecrypt wait,check,formattable,quota,latemount,resize,reservedsize=128m,checkpoint=block,fileencryption=aes-256-xts:aes-256-cts:v1

EDIT: By removing the resize flag I was able to eliminate the warning [libfs_mgr]Warning: unknown flag: resize.

lopestom commented 1 year ago

define CMD_ATTEST_KEY 24

If you enable import /vendor/etc/init/vendor.mediatek.hardware.keymaster_attestation@1.1-service.rc and

on property:hwservicemanager.ready=true start keymaster_attestation-1-1

on property:ro.crypto.state=unsupported stop keymaster_attestation-1-1

on property:ro.crypto.state=unencrypted stop keymaster_attestation-1-1

on property:twrp.decrypt.done=true stop keymaster_attestation-1-1

Anything different happen?

in your fstab: aes-256-cts:v1 But this https://github.com/ADeadTrousers/twrp_device_Unihertz_Atom_LXL/blob/e7da8613ed0871c48c4915c0e0b73fed1493a063/BoardConfigCommon.mk#L109 is better write TW_USE_FSCRYPT_POLICY := 1??

ADeadTrousers commented 1 year ago

Trust me I've tried both versions for TW_USE_FSCRYPT_POLICY multiple times and according to the make files it only affects libtar not libcrypt.

In that same matter keymaster_attestation has no counterpart on AOSP so it's more or less a vendor specific gimmick. According to the init.rc scripts from the stock rom it is only started/used during a factory reset and there is no "missing X" message in the logs so far. Nevertheless I'll try to include it to see if it changes anything at all because currently I'm taking every straw I can grab.

ADeadTrousers commented 1 year ago

As expected keymaster_attestation does nothing helpful at all.

fscrypt_initialize_systemwide_keys seems to get the wrong keys because next fscrypt_init_user0 exits with Version mismatch, expected 1 got \B3 while trying to access /data/misc/vold/user_keys/de/0/version. Therefore I assume /dava/misc/* wasn't decrypted at all by fscrypt_initialize_systemwide_keys. What could be the problem here? Incompatible encryption? The flag in fstab is fileencryption=aes-256-xts:aes-256-cts:v1 Both seem to be available according to logs:

(1)[354:recovery]fscrypt: AES-256-CTS-CBC using implementation "cts(cbc-aes-ce)"
(1)[354:recovery]fscrypt: AES-256-XTS using implementation "xts-aes-ce"

Comparing my logs from twrp with those from los the keys retrieved for /data/unencrypted/ref from keymaster seem to be identical. The one for /data/unencrypted/per_boot_ref is different though but that is to be expected. So something within fscrypt/twrp is messing things up?

lopestom commented 1 year ago

Trust me I've tried both versions for TW_USE_FSCRYPT_POLICY multiple times and according to the make files it only affects libtar not libcrypt.

In that same matter keymaster_attestation has no counterpart on AOSP so it's more or less a vendor specific gimmick. According to the init.rc scripts from the stock rom it is only started/used during a factory reset and there is no "missing X" message in the logs so far. Nevertheless I'll try to include it to see if it changes anything at all because currently I'm taking every straw I can grab.

Matthias Leitl Your attempts is with Keymaster@4.1 and so that's using keystore2? Find your stock ROM in the \system\ and let me know.

If yes so

service keystore /system/bin/keystore /tmp/misc/keystore
    user system
    group system drmrpc readproc
    disabled
    seclabel u:r:recovery:s0

Maybe not need because TeamWin placed in the \init the keystore2.rc

ADeadTrousers commented 1 year ago

There is neither a keystore2 on the stock rom nor is it being built by twrp. The only thing that kinda looks like it is keystore_cli_v2 but it's not being called by any init.rc script whatsoever. Looking at the log file keystore isn't even used. The keys seem to be stored in keyring which is started by install_keyring. Also it's NOT Keymaster@4.1. The file name AND the stock manifest.xml suggests it's 4.0. And looking into an older source code file from mediatek I managed to get my hands on around one year ago there was no mentioning of 4.1 whatsoever.

lopestom commented 1 year ago

Aaarghh... Although the recovery.img is still below the size of the partition the included ramdisk seems to be too big. As soon as it is bigger than 16.8 MB compressed or 52 MB uncompressed the device won't boot.

This took me way too long to figure out than I'm comfortable to admit.

So now I can REALLY start to work on the decryption.

Although fscrypt_initialize_systemwide_keys seems to be getting keys for decryption they don't seem to be correct as fscrypt_init_user0 fails with a version mismatch because it reads \3B whereas it should find a number at least. This means the decryption didn't work.

The error code -62 means the key need to be upgraded as the recovery runs with PLATFORM_SECURITY_PATCH := 2099-12-05 whereas system currently runs with PLATFORM_SECURITY_PATCH := 2022-12-05. So maybe something with the updgrade procedure is producing wrong keys. I'd need to tinker a bit with this.

Oh, and I neither know why the recovery is querying for 3.0::IKeymasterDevice nor why the vendor service suddenly becoming 4.1::IKeymasterDevice when it was 4.0::IKeymasterDevice during the HAL registration. But it (kinda) works. So I got this going for me which is nice.

I was just thinking about your message about keymaster3.0...... so is the first attempt to use 3.0 for the FDE and then use 4.0 for the FBE? So if the first attempt fails will the full encryption/decryption fail? I remembered this in a past experience with the code below:

Click to open

From bdcdb237bf88de33687be58ecf6e173399b33acc Mon Sep 17 00:00:00 2001 From: anonymous Date: Sat, 9 Nov 2019 19:55:54 -0600 Subject: [PATCH 06/15] Save and restore crypto footer for vold decrypt that's on a separate partition Change-Id: Id380f4fc5d4dea5fdae699eb63b3b1737dc0d503 --- crypto/vold_decrypt/vold_decrypt.cpp | 52 ++++++++++++++++++---------- 1 file changed, 33 insertions(+), 19 deletions(-) diff --git a/crypto/vold_decrypt/vold_decrypt.cpp b/crypto/vold_decrypt/vold_decrypt.cpp index ac872ea..8dc1698 100644 --- a/crypto/vold_decrypt/vold_decrypt.cpp +++ b/crypto/vold_decrypt/vold_decrypt.cpp @@ -792,7 +792,7 @@ static unsigned int get_blkdev_size(int fd) { #define CRYPT_FOOTER_OFFSET 0x4000 static char footer[16 * 1024]; -const char* userdata_path; +const char* footer_path; static off64_t offset; int footer_br(const string& command) { @@ -801,23 +801,37 @@ int footer_br(const string& command) { if (command == "backup") { unsigned int nr_sec; TWPartition* userdata = PartitionManager.Find_Partition_By_Path("/data"); - userdata_path = userdata->Actual_Block_Device.c_str(); - fd = open(userdata_path, O_RDONLY); - if (fd < 0) { - LOGERROR("E:footer_backup: Cannot open '%s': %s\n", userdata_path, strerror(errno)); + if (!userdata) return -1; - } - if ((nr_sec = get_blkdev_size(fd))) { - offset = ((off64_t)nr_sec * 512) - CRYPT_FOOTER_OFFSET; + + if (!userdata->Crypto_Key_Location.empty() + && userdata->Crypto_Key_Location != "footer") { + footer_path = userdata->Crypto_Key_Location.c_str(); + fd = open(footer_path, O_RDONLY); + if (fd < 0) { + LOGERROR("E:footer_backup: Cannot open '%s': %s\n", footer_path, strerror(errno)); + return -1; + } + offset = 0LL; } else { - LOGERROR("E:footer_br: Failed to get offset\n"); - close(fd); - return -1; - } - if (lseek64(fd, offset, SEEK_SET) == -1) { - LOGERROR("E:footer_backup: Failed to lseek64\n"); - close(fd); - return -1; + footer_path = userdata->Actual_Block_Device.c_str(); + fd = open(footer_path, O_RDONLY); + if (fd < 0) { + LOGERROR("E:footer_backup: Cannot open '%s': %s\n", footer_path, strerror(errno)); + return -1; + } + if ((nr_sec = get_blkdev_size(fd))) { + offset = ((off64_t)nr_sec * 512) - CRYPT_FOOTER_OFFSET; + } else { + LOGERROR("E:footer_br: Failed to get offset\n"); + close(fd); + return -1; + } + if (lseek64(fd, offset, SEEK_SET) == -1) { + LOGERROR("E:footer_backup: Failed to lseek64\n"); + close(fd); + return -1; + } } if (read(fd, footer, sizeof(footer)) != sizeof(footer)) { LOGERROR("E:footer_br: Failed to read: %s\n", strerror(errno)); @@ -826,12 +840,12 @@ int footer_br(const string& command) { } close(fd); } else if (command == "restore") { - fd = open(userdata_path, O_WRONLY); + fd = open(footer_path, O_WRONLY); if (fd < 0) { - LOGERROR("E:footer_restore: Cannot open '%s': %s\n", userdata_path, strerror(errno)); + LOGERROR("E:footer_restore: Cannot open '%s': %s\n", footer_path, strerror(errno)); return -1; } - if (lseek64(fd, offset, SEEK_SET) == -1) { + if (offset && lseek64(fd, offset, SEEK_SET) == -1) { LOGERROR("E:footer_restore: Failed to lseek64\n"); close(fd); return -1; -- 2.17.1

&

Click to open

From 6d45eb8b93a719f1e5eb6deb2b1a52518a267bcd Mon Sep 17 00:00:00 2001 From: anonymous Date: Sat, 9 Nov 2019 19:59:22 -0600 Subject: [PATCH 07/15] Do not use native FDE decrypt & libs if SYSTEM_VOLD is defined. Change-Id: I460663ba3bf8a40f051ed5cebfed503fe2f8cd07 --- Android.mk | 24 ++++++++++++------------ crypto/fde/Android.mk | 13 +++++++++++-- crypto/fde/cryptfs.cpp | 3 +++ partitionmanager.cpp | 3 ++- 4 files changed, 28 insertions(+), 15 deletions(-) diff --git a/Android.mk b/Android.mk index 7ffcd70..85fc212 100755 --- a/Android.mk +++ b/Android.mk @@ -323,20 +323,22 @@ ifeq ($(TW_INCLUDE_CRYPTO), true) LOCAL_CFLAGS += -DTW_INCLUDE_CRYPTO LOCAL_SHARED_LIBRARIES += libcryptfsfde libgpt_twrp LOCAL_C_INCLUDES += external/boringssl/src/include - ifeq ($(shell test $(PLATFORM_SDK_VERSION) -ge 24; echo $$?),0) - TW_INCLUDE_CRYPTO_FBE := true - LOCAL_CFLAGS += -DTW_INCLUDE_FBE - LOCAL_SHARED_LIBRARIES += libe4crypt - ifeq ($(shell test $(PLATFORM_SDK_VERSION) -ge 28; echo $$?),0) - LOCAL_CFLAGS += -DTW_INCLUDE_FBE_METADATA_DECRYPT - endif + ifeq ($(TW_CRYPTO_USE_SYSTEM_VOLD), false) + TW_CRYPTO_USE_SYSTEM_VOLD := endif - ifneq ($(TW_CRYPTO_USE_SYSTEM_VOLD),) - ifneq ($(TW_CRYPTO_USE_SYSTEM_VOLD),false) + ifeq ($(TW_CRYPTO_USE_SYSTEM_VOLD),) + ifeq ($(shell test $(PLATFORM_SDK_VERSION) -ge 24; echo $$?),0) + TW_INCLUDE_CRYPTO_FBE := true + LOCAL_CFLAGS += -DTW_INCLUDE_FBE + LOCAL_SHARED_LIBRARIES += libe4crypt + ifeq ($(shell test $(PLATFORM_SDK_VERSION) -ge 28; echo $$?),0) + LOCAL_CFLAGS += -DTW_INCLUDE_FBE_METADATA_DECRYPT + endif + endif + else LOCAL_CFLAGS += -DTW_CRYPTO_USE_SYSTEM_VOLD LOCAL_STATIC_LIBRARIES += libvolddecrypt endif - endif endif WITH_CRYPTO_UTILS := \ $(if $(wildcard system/core/libcrypto_utils/android_pubkey.c),true) @@ -869,10 +871,8 @@ ifeq ($(TW_INCLUDE_CRYPTO), true) include $(commands_TWRP_local_path)/crypto/ext4crypt/Android.mk endif ifneq ($(TW_CRYPTO_USE_SYSTEM_VOLD),) - ifneq ($(TW_CRYPTO_USE_SYSTEM_VOLD),false) include $(commands_TWRP_local_path)/crypto/vold_decrypt/Android.mk endif - endif include $(commands_TWRP_local_path)/gpt/Android.mk endif ifeq ($(BUILD_ID), GINGERBREAD) diff --git a/crypto/fde/Android.mk b/crypto/fde/Android.mk index aafd7a0..5431c10 100644 --- a/crypto/fde/Android.mk +++ b/crypto/fde/Android.mk @@ -1,5 +1,12 @@ LOCAL_PATH := $(call my-dir) ifeq ($(TW_INCLUDE_CRYPTO), true) + +ifeq ($(TW_CRYPTO_USE_SYSTEM_VOLD),) +ifeq ($(shell test $(PLATFORM_SDK_VERSION) -ge 26; echo $$?),0) + BUILTIN_CRYPTO := true +endif +endif + include $(CLEAR_VARS) LOCAL_MODULE := libcryptfsfde @@ -14,7 +21,7 @@ ifeq ($(shell test $(PLATFORM_SDK_VERSION) -lt 23; echo $$?),0) LOCAL_CPPFLAGS := -std=c++11 endif -ifeq ($(shell test $(PLATFORM_SDK_VERSION) -ge 26; echo $$?),0) +ifeq ($(BUILTIN_CRYPTO), true) #8.0 or higher LOCAL_C_INCLUDES += external/boringssl/src/include LOCAL_SHARED_LIBRARIES += libselinux libc libc++ libbase libcrypto libcutils libkeymaster_messages libhardware libprotobuf-cpp-lite libe4crypt android.hardware.keymaster@3.0 libkeystore_binder libhidlbase libutils libbinder @@ -40,6 +47,7 @@ else else LOCAL_CFLAGS += -DTW_KEYMASTER_MAX_API=0 endif + LOCAL_CFLAGS += -Wno-unused-variable -Wno-unused-parameter endif ifeq ($(TARGET_HW_DISK_ENCRYPTION),true) ifeq ($(TARGET_CRYPTFS_HW_PATH),) @@ -69,7 +77,7 @@ ifeq ($(shell test $(PLATFORM_SDK_VERSION) -lt 23; echo $$?),0) LOCAL_CPPFLAGS := -std=c++11 endif -ifeq ($(shell test $(PLATFORM_SDK_VERSION) -ge 26; echo $$?),0) +ifeq ($(BUILTIN_CRYPTO), true) #8.0 or higher LOCAL_C_INCLUDES += external/boringssl/src/include LOCAL_SHARED_LIBRARIES += libselinux libc libc++ libbase libcrypto libcutils libkeymaster_messages libhardware libprotobuf-cpp-lite libe4crypt android.hardware.keymaster@3.0 libkeystore_binder libhidlbase libutils libbinder @@ -95,6 +103,7 @@ else else LOCAL_CFLAGS += -DTW_KEYMASTER_MAX_API=0 endif + LOCAL_CFLAGS += -Wno-unused-variable -Wno-unused-parameter endif ifeq ($(TARGET_HW_DISK_ENCRYPTION),true) ifeq ($(TARGET_CRYPTFS_HW_PATH),) diff --git a/crypto/fde/cryptfs.cpp b/crypto/fde/cryptfs.cpp index 8352296..1e674bb 100644 --- a/crypto/fde/cryptfs.cpp +++ b/crypto/fde/cryptfs.cpp @@ -60,6 +60,9 @@ //#include "f2fs_sparseblock.h" //#include "EncryptInplace.h" //#include "Process.h" +#ifndef TW_KEYMASTER_MAX_API +#define TW_KEYMASTER_MAX_API 0 +#endif #if TW_KEYMASTER_MAX_API == 3 #include "../ext4crypt/Keymaster3.h" #endif diff --git a/partitionmanager.cpp b/partitionmanager.cpp index 7325fef..ec00c79 100755 --- a/partitionmanager.cpp +++ b/partitionmanager.cpp @@ -1706,6 +1706,7 @@ int TWPartitionManager::Decrypt_Device(string Password) { } int pwret = -1; +#ifndef TW_CRYPTO_USE_SYSTEM_VOLD pid_t pid = fork(); if (pid < 0) { LOGERR("fork failed\n"); @@ -1725,7 +1726,7 @@ int TWPartitionManager::Decrypt_Device(string Password) { pwret = WEXITSTATUS(status) ? -1 : 0; } -#ifdef TW_CRYPTO_USE_SYSTEM_VOLD +#else if (pwret != 0) { pwret = vold_decrypt(Password); switch (pwret) { -- 2.17.1

I'm writing this because on many stock ROMs under \system\lib64 I've seen the decrypt.so or libdecrypt.so file, but I haven't seen that included in TWRP. Maybe it's embedded in another file, but even then it's just a comment to think about. Ah! Remenbering that's about this:

TW_CRYPTO_USE_SYSTEM_VOLD := servicemanager hwservicemanager keymaster-3-0
TW_CRYPTO_SYSTEM_VOLD_MOUNT := vendor

But in actual time changing keymaster-3-0 to keymaster-4-0 can help in anything?

ADeadTrousers commented 1 year ago

I've already tried to use vold but wasn't able to get it to compile (some header files cannot be found and so on) so I assume it wasn't planned/tested to be useable for twrp-11. AFAIK starting from twrp-11 when they switched from omni to aosp they included all of vold's functionality into the recovery.

For the both changes/patches can you get me the urls (github, gerrit, ...)? The code here in text format is a little bit hard for me to read and understand.

lopestom commented 1 year ago

I've already tried to use vold but wasn't able to get it to compile (some header files cannot be found and so on) so I assume it wasn't planned/tested to be useable for twrp-11. AFAIK starting from twrp-11 when they switched from omni to aosp they included all of vold's functionality into the recovery.

For the both changes/patches can you get me the urls (github, gerrit, ...)? The code here in text format is a little bit hard for me to read and understand.

ext4crypt: change to upgrade key if export fails ext4crypt: support wrappedkey for FBE https://github.com/PitchBlackRecoveryProject/android_bootable_recovery/commit/821d7b554cb94d9d270dc7539c9777e64b3033bc

https://gerrit.omnirom.org/c/android_bootable_recovery/+/21050/

https://gerrit.omnirom.org/c/android_bootable_recovery/+/32481 https://gerrit.omnirom.org/c/android_bootable_recovery/+/32689/

https://github.com/rezaadi0105/omnirom_bootable_recovery/blob/android-9.0/crypto/ext4crypt/KeyStorage4.cpp

I forgot about the speciallist....... Maybe @bkerler can help........

ADeadTrousers commented 1 year ago

FDE isn't part of twrp-11 anymore. Same for EXT4CRYPT. As I already mentioned VOLD_DECRYPT isn't builtable although it is mentioned as a fallback option back in 2017 when twrp was still based on omnirom. That is the thing: twrp-10 which was never officially released and is now deprecated worked fine with the decryption as I suspect it used vold_decrypt.

lopestom commented 1 year ago

That's more than I can undertand What need. https://android.googlesource.com/platform/hardware/interfaces/+/master/keymaster/4.0/types.hal https://source.android.com/docs/security/features/keystore/tags

I tried understand abou e4crypt in the old version and that using keymaster4. But i not know if help.... https://github.com/TeamWin/Team-Win-Recovery-Project/blob/58f2132bc3954fc704787d477500a209eedb8e29/libtar/Android.mk#L16

About trustkernel/tkcore: https://github.com/erfanoabdi/android_kernel_gigaset_mt6768/commit/9ae0cdaafdaedd038176208cadd302d7a49252c6

I think about placing the sepolicy_vndr in the DT but I not know if is good. https://github.com/schoeller/android_device_mediatek_sepolicy_vndr/tree/lineage-18.1/bsp/non_plat You can see about tkcore-teed-attestation-mobicore-others......... https://github.com/schoeller/android_device_mediatek_sepolicy_vndr/blob/lineage-18.1/bsp/non_plat/teed_app.te https://github.com/schoeller/android_device_mediatek_sepolicy_vndr/blob/lineage-18.1/bsp/non_plat/tkcore_daemon.te https://github.com/schoeller/android_device_mediatek_sepolicy_vndr/blob/lineage-18.1/bsp/non_plat/storageproxyd.te

This change keys? https://github.com/schoeller/android_device_mediatek_sepolicy_vndr/blob/02ffd17bd948ca72927f9d0c81cb4a01c952fe13/bsp/non_plat/keystore.te#L22

Do you can test putting setprop in the init.recovery.device.rc these:?

on fs
setprop sys.usb.config adb
install_keyring

or

on boot
    setprop sys.usb.config adb

Only for test what happen......

lopestom commented 1 year ago

Forgot mention about setprop. But about other old message: https://github.com/brigudav/device_xiaomi_zeus_twrp/blob/8613b37fb9093e231df4c374c77e219b05966a08/recovery/root/init.recovery.qcom.rc#L16

Ah! Other question is about this: https://github.com/ADeadTrousers/twrp_device_Unihertz_Atom_LXL/blob/c42a183b5a0b642e561eeec66d1cc80e9cbb21d3/recovery/root/vendor/etc/init/trustkernel.rc#L70

That's running correctly?

I finded this in other DT: https://github.com/LuciferMorningStar733/device_xiaomi_pissarro/blob/c06022b798b3346eaf61ad14bf1eef8e47bdbce3/recovery/root/microtrust.rc#L233

lopestom commented 1 year ago

I don't know if this file is in your ROM in \system\system\bin\gatekeeperd but I've seen it in many ROMs on Android 11. I wondered why it exists and what it's for. Maybe it's good to leave this as a good read, since in Android 14 we will again have another form of encryption. gatekeeperd (Gatekeeper daemon). Gatekeeper Working

https://source.android.com/docs/security/features/authentication/gatekeeper#architecture

https://source.android.com/docs/security/features/encryption/file-based#encryption-policy

https://source.android.com/docs/security/features/encryption/metadata#prerequisites https://source.android.com/docs/security/features/encryption/metadata#changes-to-the-init-sequence https://source.android.com/docs/security/features/encryption/metadata#common-issues

https://source.android.com/docs/security/features/encryption/metadata#configuration-on-adoptable-storage-legacy

I have a question in your fstab. For what reason your put in the twrp.fstab

# Core Partitions
/md_udc                 emmc    /dev/block/by-name/md_udc                               flags=backup=1
/metadata               emmc    /dev/block/by-name/metadata                             flags=backup=1

so you not put this like your stock recovery.fstab?? /metadata ext4 /dev/block/by-name/md_udc

ADeadTrousers commented 1 year ago

Hi. I've gone through your suggestions. I'm not going into details on every bit but here are some comments on the more juicier parts. 1) I won't switch to a custom kernel because this will involve too much work on my side. 2) The vendor sepolicies are not needed (for now?) because all involved programs are run in the "recovery" context. 3) setprop sys.usb.config adb does nothing on decryption. It's for usb configuration. 4) twrp isn't using vold and trying to activate it in the built results in dependencies not met. 5) tee_check_keybox is for checking something I don't know yet. It tries to read /data/vendor/t6/tkcore.log but as this is still encrypted on execution I assume it's not vital for the decryption itself. 6) As long as there is no newer kernel from unihertz the newer decryption won't be possible to include. 7) I've taken some parts of my twrp.fstab from other device trees but those settings are for twrp only. The decryption is using recovery.fstab.

ADeadTrousers commented 1 year ago

Hi @lopestom About your invitation to https://github.com/lopestom/android_hardware_mediatek What is it? On first sight it looks like a bootloader.

lopestom commented 1 year ago

Hi @lopestom About your invitation to https://github.com/lopestom/android_hardware_mediatek What is it? On first sight it looks like a bootloader.

https://github.com/lopestom/android_hardware_mediatek/wiki/analisys-rpmb

ADeadTrousers commented 1 year ago

I think I'm on the right way now. By renaming twrp.fstab to twrp.flags in 8c4e674a6a9634af1c10e3b2d034c53744750961 I'm now facing the pattern/password screen in twrp.

It's still not letting me decrypt my storage though.

According to /tmp/recovery.log it's able to read the pwd file in /data/system_de/0/spblob/ at least once which means the system decryption was sucessful:

Using synthetic password method
Handle is '306d8f55647a3c6a'
password type: pattern
I:PageManager::LoadFileToBuffer loading filename: '/data/system/users/0.xml' directly
Using synthetic password method
Handle is '306d8f55647a3c6a'
password type: pattern
I:User 0 is not decrypted.

Next it's preparing the decrypt and decrypt_pattern screens:

I:Unmounting main partitions...
I:Is encrypted, do decrypt page first
I:Switching packages (TWRP)
I:Set page: 'decrypt'
I:Set page: 'decrypt_pattern'
I:Found a match 'mmcblk0' '/devices/platform/externdevice'
I:Decrypt adopted storage starting
I:PageManager::LoadFileToBuffer loading filename: '/data/system/storage.xml' directly
I:No /data/system/storage.xml for adopted storage
I:No adopted storage so finding actual block device
...
I:Set page: 'trydecrypt'
I:operation_start: 'Decrypt'

But now it's not able to read the password file any more as it's attempting to fallback on gatekeeper:

Attempting to decrypt FBE for user 0...
Unable to locate gatekeeper password file '/data/system/gatekeeper.pattern.key'
Unknown password type
Failed to decrypt user 0
I:Set page: 'decrypt'
I:Set page: 'decrypt_pattern'
I:operation_end - status=1
I:Set page: 'trydecrypt'
I:operation_start: 'Decrypt'
Attempting to decrypt FBE for user 0...
Unable to locate gatekeeper password file '/data/system/gatekeeper.pattern.key'
Unknown password type
Failed to decrypt user 0
I:Set page: 'decrypt'
I:Set page: 'decrypt_pattern'
I:operation_end - status=1
I:Set page: 'canceldecrypt'

According to the source code in Decrypt.cpp the function Get_Password_Type needs access to /data/system_de/0/spblob/ in order to be able to read the pwd file. Therefore the system decryption is lost during the startup of the recovery.

My best guess would be an unmount or something similar between the the first successful read an the pattern read.

lopestom commented 1 year ago

I think I'm on the right way now. By renaming twrp.fstab to twrp.flags in 8c4e674 I'm now facing the pattern/password screen in twrp.

My best guess would be an unmount or something similar between the the first successful read an the pattern read.

Why did you not include \vintf folders; manifest and \vendor\etc\uevent.rc files?

ADeadTrousers commented 1 year ago

I think I'm on the right way now. By renaming twrp.fstab to twrp.flags in 8c4e674 I'm now facing the pattern/password screen in twrp. My best guess would be an unmount or something similar between the the first successful read an the pattern read.

Why did you not include \vintf folders; manifest and \vendor\etc\uevent.rc files?

Because they are part of the stock rom, don't need to be changed at all and therefore copied directly from the stock rom/the device: https://github.com/ADeadTrousers/twrp_device_Unihertz_Atom_LXL/blob/master/proprietary-files.txt

ADeadTrousers commented 1 year ago

The fstab I use for recovery is the same I use for system. Therefore I don't think I should have it differ too much as these settings are crucial for the first time initialization and also for every decryption attempt afterwards.

metadata is not required for my device. Otherwise the fstab would look similar to this https://github.com/pjgowtham/recovery_device_xiaomi_pissarro/blob/master/recovery.fstab

TWRP for Android 11 and up does not include vold. It's functionality is integrated into the recovery binary. The same goes for wait_for_keymaster. In fact keymaster is woking fine because otherwise the recovery would not have figured out that I use a pattern for my password. It's just that later on in the boot process the decryption is somehow "lost".

ADeadTrousers commented 1 year ago

Something in TWPartitionManager::Setup_Fstab_Partitions seems to be not working correctly (at least for my device). I just tried to decrypt without any password or pattern set and it's still not able to decrypt.

To make sure that no "switch" to a different part of the recovery (e.g. to the decrypt_screen) is interfering with the overall decryption I identified this function to be solely responsible for the overall process (at least for default password) by dissecting the recovery.log.

I:File Based Encryption is present
SELinux: Loaded file_contexts
Using synthetic password method
Handle is '745469b84c431db0'
using default password
I:PageManager::LoadFileToBuffer loading filename: '/data/system/users/0.xml' directly
Using synthetic password method
Handle is '745469b84c431db0'
using default password
I:User 0 is not decrypted.
I:Setting up '/data' as data/media emulated storage.
I:unable to open zip archive /system_root/system/apex/com.google.android.tzdata2.apex. Reason: No such file or directory
I:Skipping non-existent apex file: /system_root/system/apex/com.google.android.tzdata2.apex
I:unable to open zip archive /system_root/system/apex/com.google.android.media.swcodec.apex. Reason: No such file or directory
I:Skipping non-existent apex file: /system_root/system/apex/com.google.android.media.swcodec.apex
I:Backup folder set to '/data/media/0/TWRP/BACKUPS/Atom_XL'
I:Settings storage is '/data/media/0'
Attempting to decrypt FBE for user 0...
Unable to locate gatekeeper password file '/data/system/gatekeeper.pattern.key'
unlock_user_key returned fail
Failed to decrypt user 0
Unable to decrypt with default password. You may need to perform a Format Data.

The first few lines are from calling Partition_Post_Processing on line 285 of partitionmanager.cpp The lines regarding apex are from calling loadApexImages on line 321 of partitionmanager.cpp (this can be deactivated with TW_EXCLUDE_APEX) The line I:Settings storage is '/data/media/0' is from calling Setup_Settings_Storage_Partition on line 374 of partitionmanager.cpp The rest is from the call of Decrypt_Data on line 379 of partitionmanager.cpp.

So that means between the first two attempts to read the password type where it was able to read the system encrypted info (using default password) and the actual decryption of user 0 the system encryption is somehow "lost". Therefore it's not able to read the password type and defaults to gatekeeper.

ADeadTrousers commented 1 year ago

Eureka! TWPartition::Setup_Data_Media is the culprit. Or more specifically the UnMount(true) in line 1240 in partition.cpp. After that the system decryption is removed. As far as I can tell this code should relink /data/media/0 to /data.

ADeadTrousers commented 1 year ago

It looks like the guys at TeamWin already fixed that ... in android-12.1 https://github.com/TeamWin/android_bootable_recovery/blob/android-12.1/partition.cpp#L1252

ADeadTrousers commented 1 year ago

https://github.com/ADeadTrousers/twrp_device_Unihertz_Atom_LXL/blob/master/patch/bootable_recovery/0002-decryption-solve-specific-decryption-problems-on-som.twrp.patch

With this patch I'm now able to at least decrypt /data if no password or pattern is set. Trying to decrypt with a password or pattern results in an endless loop. So there is still some tweaking need, but that's for another day.

ADeadTrousers commented 1 year ago

I just had to revert some changes I did in init.recovery.trustkernel.rc and now also the decryption of a pattern or password encrypted user works as it should.

lopestom commented 1 month ago

Hi @ADeadTrousers, I read from xda about decryption not working because big size file than 32MB. You can test compiling without some tools. Like this: https://github.com/lopestom/twrp_device_ulefone_Power_Armor14_Pro/releases

flags in here

You can use the script for removing stock recovery if you want. That already work for other device. I not remember if device need in RW. Test and let me know. system-bin-postrecoveryboot.zip

You can add too if decrypt not work: BoardConfig

# Crypto
TW_FORCE_KEYMASTER_VER := true

system.prop

keymaster_ver=4.0
ADeadTrousers commented 1 month ago

The TW_EXCLUDE_TZDATA := true looks VERY promising. I'll have to try that. Thanks.

ADeadTrousers commented 1 month ago

TW_EXCLUDE_TZDATA := true did work so no traces of tzdata can be found in the resulting ramdisk. BUT Even enabling all TW_EXCLUDE_* switches and not including any aditional vendor blobs for the decryption still gives me a ramdisk of around 19MB (compressed) where I'd need something around 16~17MB (vendor blobs included). Not to mention the recovery is not bootable with this big chunk of a ramdisk.

lopestom commented 1 month ago

..... still gives me a ramdisk of around 19MB (compressed) where I'd need something around 16~17MB (vendor blobs included). Not to mention the recovery is not bootable with this big chunk of a ramdisk.

Doubts: 1- Your device update to A12? 2- If not so What reason to have twrp-12.1? Maybe more compatibility with LOS-2x? 3- The problem in actual branch twrp-12.1 from TeamWin is about new updates and generally the more files. One problem to old devices shipped with < A11 is not using keymint mode. So if you unpacking img file and look \system\lib64 & \system\bin so you find and can know the sizes of these files you can undertand about. 3.1- But if you look my repo UPA14Pro so you you will see that the img file was compiled before many changes. So maybe if you manage to have the twrp-12.1 branch repository before these changes (repopick?), I believe you will still have the possibility of reducing the size of the img file. Well, if we could change something in the TeamWin source code so that legacy devices on A12 do not contain the keymint files and their dependencies, then it would be even more interesting. These devices use keymaster4_4.1 encryption and thus would not need keymint. I think it works that way, but I don't know if this will be a solution.

4- Generally trustkernel mode has large files compared to microtrust and trustonic. So this is also something that gets in the way. If we could know whether this or that file with large size would not be useful for decryption, then it would be more interesting. If not, it really is a huge dependency.

Finally, what I find strange is that I did a recovery.img on twrp-12.1 with trustkernel mode and the img file was compiled. When unpacking and repackaging, the img file went from 32MB to 29.8MB. Why didn't this happen to you?

Comparinng the kernel file by Atom so 9.54 MB so has some strange thing than mine.....

File Size
recovery-A12-20240524-Crypt.img-kernel 10.1 MB
recovery-A12-20240524-Crypt.img-dtb 109.5 KB
recovery-A12-20240524-Crypt.img-recovery_dtbo 53.1 KB

Others files tthat you see:

Pictures - Click to open

![ksnip_20240601-215809](https://github.com/ADeadTrousers/twrp_device_Unihertz_Atom_LXL/assets/25868008/79e62eb5-d236-462b-b99b-390a21069420) ![ksnip_20240601-215225](https://github.com/ADeadTrousers/twrp_device_Unihertz_Atom_LXL/assets/25868008/9e977a7b-93d4-4591-ab5a-db8ae5804561) ![ksnip_20240601-215256](https://github.com/ADeadTrousers/twrp_device_Unihertz_Atom_LXL/assets/25868008/4662db54-dd14-4ac6-8047-5fc0230e1062)

ADeadTrousers commented 1 month ago

ad 1) Vendor/Kernel is A11. LOS/System is A12. As A11 stopped getting security updates I tried to switch to A12 and it worked (at least with LOS). Now I'm trying to get decryption to work (again). ad 2) The decryption uses system properties(?) to define the cypher. So in order to decrypt A12 devices I need a A12 recovery. A11 does not work (belief me I tried). ad 3 & 3.1) That's very informative thanks. ad 4) The vendor blobs aren't that big when compressed (around 1MB if I remeber correctly) and I identified the dependencies to the T6-blobs so I was able to reduce them even further to the bare minimum and leave out unnecessary ones. The problem is, even without these blobs the ramdisk is still too big to load.

The thing with 32MB vs. 29.8MB is that I need a recovery.img with the size of 32MB to properly load. Therefore I'm required to set BOARD_RECOVERYIMAGE_PARTITION_SIZE to 32MB to create a bigger file. The additional 2.2MB are just "empty" space that gets added (I think).

There are two particular huge files libicui18n.so (2.6MB) and libicuuc.so (1.8MB) that were added with A12 and are soley responsible for the increase of the lib64 folder betweeen A11 and A12. I think they are linked with tzdata. That's the reason I want to get rid of it entirely. Also nearly every main file in bin doubled in size (recovery 1.1MB -> 2.3MB; adbd 1.4MB -> 2MB; init 1.1MB -> 1.9MB; fastbootd 0.2MB -> 1.2MB) and keystore2 got added (1.5MB)

So these are my main concerns right now: To somehow get rid of tzdata entirely(!), reduce the size of the bin files (e.g. with global compiler optimizations) and remove keystore2 from the build.

Of course I could try that by modifying the resulting ramdisk but I want to be able to do that during building of TWRP and therefore be able to reproduce it later when security updates need to be applied or when upgrading (e.g. to A13 if that's ever going to happen).