search

Halium / projectmanagement

143 stars 32 forks source link

[device-port] [mata] Port to Essential Ph-1 #122

Open xonq opened 5 years ago

xonq commented 5 years ago

Tree:

xonq commented 5 years ago

Following enabling envsetup.sh, I get this error 'Can not locate config makefile for product "lineage_mata". It exists in BUILDDIR/halium/device/essential/mata.

`breakfast mata including vendor/cm/vendorsetup.sh build/core/product_config.mk:249: Can not locate config makefile for product "lineage_mata". Stop. build/core/product_config.mk:249: Can not locate config makefile for product "cm_mata". Stop. build/core/product_config.mk:249: Can not locate config makefile for product "lineage_mata". Stop. Device mata not found. Attempting to retrieve device repository from LineageOS Github (http://github.com/LineageOS). Found repository: android_device_essential_mata Traceback (most recent call last): File "vendor/cm/build/tools/roomservice.py", line 263, in default_revision = get_default_revision() File "vendor/cm/build/tools/roomservice.py", line 117, in get_default_revision return r.replace('refs/heads/', '').replace('refs/tags/', '') AttributeError: 'NoneType' object has no attribute 'replace' build/core/product_config.mk:249: Can not locate config makefile for product "lineage_mata". Stop. build/core/product_config.mk:249: *** Can not locate config makefile for product "lineage_mata". Stop.

Don't have a product spec for: 'lineage_mata' Do you have the right repo manifest?`

xonq commented 5 years ago

Attempting to setup remote for unofficial lineage 16.0, Invisiblek's, in the manifest.

xonq commented 5 years ago

'breakfast' command still failing - it isnt finding my lineage_mata.mk file even though it exists

jbruechert commented 5 years ago

Can you post a full log? Maybe some dependency makefile is missing.

xonq commented 5 years ago 
including vendor/cm/vendorsetup.sh
build/core/product_config.mk:249: *** Can not locate config makefile for product "lineage_mata".  Stop.
build/core/product_config.mk:249: *** Can not locate config makefile for product "cm_mata".  Stop.
build/core/product_config.mk:249: *** Can not locate config makefile for product "lineage_mata".  Stop.
Device mata not found. Attempting to retrieve device repository from LineageOS Github (http://github.com/LineageOS).
Found repository: android_device_essential_mata
Default revision: lineage-16.0
Checking branch info
Checking if device/essential/mata is fetched from android_device_essential_mata
LineageOS/android_device_essential_mata already fetched to device/essential/mata
Syncing repository to retrieve project.
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0        
fatal: remote github not defined in /home/xaq/halium/.repo/manifest.xml
Repository synced!
Looking for dependencies in device/essential/mata
Done
build/core/product_config.mk:249: *** Can not locate config makefile for product "lineage_mata".  Stop.
build/core/product_config.mk:249: *** Can not locate config makefile for product "lineage_mata".  Stop.

** Don't have a product spec for: 'lineage_mata'
** Do you have the right repo manifest?

I put the dependencies from lineagedependencies into the manifest. cm_mata doesn't exist, but lineage_mata does in halium/device/essential/mata

xonq commented 5 years ago 
~/halium$ breakfast
including vendor/cm/vendorsetup.sh

You're building on Linux

Breakfast menu... pick a combo:
 1. full-eng 
... and don't forget the bacon!

Which would you like? [aosp_arm-eng]

Here's my breakfast command output as well

jbruechert commented 5 years ago

Are you trying to use a lineage-16.0 device tree? The halium tree is currently 14.1 based, so this will not work!

xonq commented 5 years ago

Worked

ghost commented 5 years ago

I'm not sure if you've had any luck, but I was able to get both the hybris-boot and system images built, but have run into some issues getting them installed because of the slots

alefnode commented 5 years ago

I found a problem when make mka hybris-boot

[ 32% 675/2049] Building Kernel Config FAILED: /bin/bash -c "(make -j8 CFLAGS_MODULE=\"-fno-pic\" -C kernel/essential/msm8998 O=/home/debian/halium/out/target/product/mata/obj/KERNEL_OBJ ARCH=arm64 CROSS_COMPILE=\" aarch64-linux-androidkernel-\" VARIANT_DEFCONFIG= SELINUX_DEFCONFIG= lineageos_mata_defconfig ) && (if [ ! -z \"\" ]; then echo \"Overriding kernel config with ''\"; echo >> /home/debian/halium/out/target/product/mata/obj/KERNEL_OBJ/.config; make -j8 -C kernel/essential/msm8998 O=/home/debian/halium/out/target/product/mata/obj/KERNEL_OBJ ARCH=arm64 CROSS_COMPILE=\" aarch64-linux-androidkernel-\" oldconfig; fi ) && (make -j8 -C kernel/essential/msm8998 O=/home/debian/halium/out/target/product/mata/obj/KERNEL_OBJ ARCH=arm64 CROSS_COMPILE=\" aarch64-linux-androidkernel-\" savedefconfig ) && (if [ ! -z \"\" ]; then echo \"Using additional config ''\"; kernel/essential/msm8998/scripts/kconfig/merge_config.sh -m -O /home/debian/halium/out/target/product/mata/obj/KERNEL_OBJ /home/debian/halium/out/target/product/mata/obj/KERNEL_OBJ/.config kernel/essential/msm8998/arch/arm64/configs/; make -j8 -C kernel/essential/msm8998 O=/home/debian/halium/out/target/product/mata/obj/KERNEL_OBJ ARCH=arm64 CROSS_COMPILE=\" aarch64-linux-androidkernel-\" KCONFIG_ALLCONFIG=/home/debian/halium/out/target/product/mata/obj/KERNEL_OBJ/.config alldefconfig; fi )" make: se entra en el directorio '/home/debian/halium/kernel/essential/msm8998' make[1]: se entra en el directorio '/home/debian/halium/out/target/product/mata/obj/KERNEL_OBJ' HOSTCC scripts/basic/fixdep GEN ./Makefile HOSTCC scripts/kconfig/conf.o SHIPPED scripts/kconfig/zconf.tab.c SHIPPED scripts/kconfig/zconf.lex.c SHIPPED scripts/kconfig/zconf.hash.c HOSTCC scripts/kconfig/zconf.tab.o HOSTLD scripts/kconfig/conf drivers/staging/Kconfig:113: can't open file "drivers/staging/qcacld-3.0/Kconfig" /home/debian/halium/kernel/essential/msm8998/scripts/kconfig/Makefile:110: fallo en las instrucciones para el objetivo 'lineageos_mata_defconfig' make[2]: [lineageos_mata_defconfig] Error 1 /home/debian/halium/kernel/essential/msm8998/Makefile:535: fallo en las instrucciones para el objetivo 'lineageos_mata_defconfig' make[1]: [lineageos_mata_defconfig] Error 2 make[1]: se sale del directorio '/home/debian/halium/out/target/product/mata/obj/KERNEL_OBJ' Makefile:146: fallo en las instrucciones para el objetivo 'sub-make' make: [sub-make] Error 2 make: se sale del directorio '/home/debian/halium/kernel/essential/msm8998' [ 32% 675/2049] target asm: libc_syscalls <= bionic/libc/arch-arm64/syscalls/_exit.S ninja: build stopped: subcommand failed. build/core/ninja.mk:151: fallo en las instrucciones para el objetivo 'ninja_wrapper' make: [ninja_wrapper] Error 1 make: se sale del directorio '/home/debian/halium'

Anyone else with this error?

alefnode commented 5 years ago

Some tips: sudo apt install libssl-dev

Comment line 113 from ./kernel/essential/msm8998/drivers/staging/Kconfig

source "drivers/staging/qcacld-3.0/Kconfig"

ghost commented 5 years ago

Yeah there are definitely a couple weird little things about the build. With those two tips, were you able to finish the hybris-boot build?

alefnode commented 5 years ago

Yeah "mka hybris-boot" works with that tips, now i'm stacked on "mka systemimage" with this error:

Running kati to generate build-lineage_mata.ninja... No need to regenerate ninja file Starting build with ninja ninja: Entering directory `.' ninja: error: '/home/debian/halium/out/host/linux-x86/bin/build_verity_tree', needed by '/home/debian/halium/out/target/product/mata/obj/PACKAGING/systemimage_intermediates/system.img', missing and no known rule to make it build/core/ninja.mk:151: fallo en las instrucciones para el objetivo 'ninja_wrapper' make: *** [ninja_wrapper] Error 1 make: se sale del directorio '/home/debian/halium'

alefnode commented 5 years ago

Modify file: build/target/product/verity.mk Set PRODUCT_SUPPORTS_VERITY := false And delete verity_key from PRODUCT_PACKAGES

alefnode commented 5 years ago

It works! you can use my last tips to compile halium on essential phone

alefnode commented 5 years ago

Ok, after flashing hybris-boot and system.img i've got a bootloop, y tried to debug it but i can't

alefnode commented 5 years ago

https://paste.ubuntu.com/p/VSH3dKhgzd/

Maxr1998 commented 5 years ago

The reboot/bootloop seems to be because of dm-verity, did you try to disable that? Afaik you have to remove the verify mount option inside the fstab.

xonq commented 5 years ago

Will this manifest work - it's based on the 16.0 dependencies. If you look at invisiblek's cm-14.1 lineageos repo, the dependencies are much more extensive. I'm going to bake that into my manifest and work from that.

xonq commented 5 years ago

my systemimage keeps failing with

warning: (SND_SOC_WCD934X) selects SND_SOC_WCD_SPI which has unmet direct dependencies (SOUND && !M68K && !UML && SND && SND_SOC && CONFIG_SPI) warning: (SND_SOC_WCD934X) selects SND_SOC_WCD_SPI which has unmet direct dependencies (SOUND && !M68K && !UML && SND && SND_SOC && CONFIG_SPI)

xonq commented 5 years ago

fixed and successfully built systemimage by running make clean after building the boot img and used export USE_HOST_LEX=yes

disabled 'verify' in the fstab and set CONFIG_DEFAULT_SECURITY to apparmor in lineageos_mata_defconfig, also adjusted CONFIG_SECURITY_xx_BOOTPARAM to 1 for SELINUX and 0 for apparmor

will flash soon and report back

xonq commented 5 years ago

Edit 190712: I believe to be caused by a mistake building, so this may be ignorable

kernel panic caused by SE Linux: https://paste.ubuntu.com/p/dGDsPmxjPv/ (please note I'm working to boot ubuntu touch so am modding files with their tools)

here are my defconfig se linux parameter values: CONFIG_SECURITY_SELINUX_BOOTPARAM_VALUE=1 CONFIG_SECURITY_SELINUX=y CONFIG_SECURITY_SELINUX_BOOTPARAM=y CONFIG_SECURITY_SELINUX_DISABLE=y CONFIG_SECURITY_SELINUX_DEVELOP=y CONFIG_SECURITY_SELINUX_AVC_STATS=y CONFIG_SECURITY_SELINUX_CHECKREQPROT_VALUE=1

I rebuilt halium-boot after changing the bootparam, selinux, develop, and stats to n. Caused a watchdog bite after flashing, otherwise output was the same.

alefnode commented 5 years ago

Did you see this?

[ 1.696244] init: No precompiled sepolicy: No such file or directory [ 1.696255] init: Compiling SELinux policy [ 1.696386] init: Failed to read /vendor/etc/selinux/plat_sepolicy_vers.txt: No such file or director

xonq commented 5 years ago

That's exactly what I saw before the panic

ub ports does not use se linux though, correct? but should I set the default security parameter from apparmor back to selinux?

alefnode commented 5 years ago

Are you finally working with cm14.1?

Can you share your manifest?

alefnode commented 5 years ago

How did you disable verify?, i can't pass over that point

xonq commented 5 years ago

I'm using the same manifest as provided at this point. I will begin working with invisiblek's cm-14.1 dependencies if these dont follow through. I have disabled verify, will flash tonight to check - ran into some issues flashing.

Got to have a USB 2 to C cable and port, the right fastboot, and all that

xonq commented 5 years ago

I disabled verify per the advice above. Edit: /halium/device/essential/mata/rootdir/root/fstab.mata From there you will see a parameter on the first line of sources that says verify, simply delete that

xonq commented 5 years ago

I disabled verify and got the same error you were receiving alefnode about dm-verity failing. Went into the defconfig and commented out 3 lines dealing with verity. The new error is something to do with dm-0 Edit: see alefnode's paste 7 comments down for details. EDIT: built from the old dependencies in lineagos-cm-14.1 (largely @invisiblek's builds and hardware trees successfully below

alefnode commented 5 years ago

I made some changes: File ./device/essential/mata/device.mk comment lines

PRODUCT_SYSTEM_VERITY_PARTITION := /dev/block/bootdevice/by-name/system

$(call inherit-product, build/target/product/verity.mk)

File ./kernel/essential/msm8998/arch/arm64/configs/mata_defconfig CONFIG_DM_VERITY=n CONFIG_DM_VERITY_FEC=n CONFIG_DM_ANDROID_VERITY=n

But still not working

xonq commented 5 years ago

I made some changes: File ./device/essential/mata/device.mk comment lines

PRODUCT_SYSTEM_VERITY_PARTITION := /dev/block/bootdevice/by-name/system

$(call inherit-product, build/target/product/verity.mk)

File ./kernel/essential/msm8998/arch/arm64/configs/mata_defconfig CONFIG_DM_VERITY=n CONFIG_DM_VERITY_FEC=n CONFIG_DM_ANDROID_VERITY=n

But still not working

have you tried messing with the lineage mata defconfig? edit: /halium/kernel/essential/msm8998/arch/arm64/configs/lineageos_mata_defconfig

alefnode commented 5 years ago

No, i am going to did it now, for checking

alefnode commented 5 years ago

I tried all, all the change commented on this post, but i continue with same error

Fstab + device.mk + verity.mk + defconfig

https://paste.ubuntu.com/p/7FTjQJ6FzF/

xonq commented 5 years ago

did you verify that the variables were actually changed in the final .config file? Should be located in the /out folder I'm guessing

alefnode commented 5 years ago 
Yeah, i have this

fs_type=ext4
system_size=4294967296 
extfs_sparse_flag=-s
squashfs_sparse_flag=-s
selinux_fc=/home/debian/halium/out/target/product/mata/root/file_contexts.bin
recovery_as_boot=true
system_root_image=true
ramdisk_dir=/home/debian/halium/out/target/product/mata/root
skip_fsck=true

Ignore this comment, i think i was using wrong defconfig file, testing again.

alefnode commented 5 years ago

Ok, finally i pass to next step:

https://paste.ubuntu.com/p/fV597YgDGx/

xonq commented 5 years ago

I commented out the verity options in device.mk in the /halium/build directory and rebuilt. Will update later

xonq commented 5 years ago

That didn't work. Perhaps we should be focusing on making verity work instead of disabling it.

xonq commented 5 years ago

I'm failing at building build_verity_tree for systemimage. I'm not sure how to build that binary. mka build_verity_tree doesn't work

Maxr1998 commented 5 years ago

It seems you have to build that one "manually" with cryptsetup, as it's described here: https://source.android.com/security/verifiedboot/dm-verity#hash-tree

I love what you guys are doing here, and hope you'll get it to work eventually. I'd contribute if my PH-1 wasn't my daily driver..

xonq commented 5 years ago

Perfect, will follow up with that. I started using the lineage-dependencies from the cm-14.1 tree for my repo manifest because I noticed quite a few hardware connections were failing in the boot notes. @Invisiblek has specific hardware dependencies in his repository in the cm-14.1 tree. Edit: The only difference in building is that the opensource-nfc.. (near field communication) module fails to build due to missing a java_library, so I omitted that at this point. I assume this manifest will be better because it specifies more dependencies directly... the qcacld3.0 built, which was an error in the previous manifest.

Edit: cm 14.1 manifest: https://pastebin.com/2zqCT2VM

xonq commented 5 years ago

I tried following those instructions to build and sign verity, but it wasn't working for the boot.img successfully. At this time, I found a simple patch for the verity option I'm going to try.

alefnode commented 5 years ago

Awesome manifest, i would try that too.

The patch is same thing that we mention before, so i think it is not going to work

xonq commented 5 years ago

You're right, it didnt. So, as I see it:

FYI if you're having trouble with twrp touch functionality you have to:

Reboot fastboot run flashall from factory image download Flash twrp.img to ONE slot Change to slot you just flashed, reboot recovery run adb shell twrp install twrp-installer.zip, Reboot fastboot Change to other slot, twrp is functional on this slot

alefnode commented 5 years ago

Yeah, i have touch working at first i was crazy about that.

Do you think this can help us? https://forum.xda-developers.com/android/software-hacking/signing-boot-images-android-verified-t3600606

xonq commented 5 years ago

I tried that and it didnt work, however I changed the key name to verity. Might be worth you trying!

ghost commented 5 years ago

So I ended up cheating a bit and just extracted the tarball version of this repo into the out/host/linux-x86/bin folder and renamed build_verity_tree.py to build_verity_tree https://android.googlesource.com/platform/system/extras/+/724396ed8dbf71ce2744e916fec2e6aec05d1041/verity/

Good news is that it worked well enough to not fail there again. Bad news is it's then failing and calling for a new binary file fec.

I haven't been able to find that yet, but if I can, I'll see what happens with the same method.

Potentially also of note, I found Essential's Github page: https://github.com/EssentialOpenSource where they have a repo for qcacld-3.0

xonq commented 5 years ago

@laserbeamh great news, that's a significant hurdle overcome.

I've reverted back to Essential being my daily driver, but I can do some research in the meantime before I move forward with development again.

xonq commented 5 years ago

Perhaps this will work.

ghost commented 5 years ago

took the lazy route (again). eventually i'll go back and at least set git dependencies correctly.

this is the repo for fec: https://android.googlesource.com/platform/system/extras/+/master/verity/fec/tests/

mka systemimage failed at 7%

[ 7% 1572/22028] Lex: libsepol_32 <= external/selinux/libsepol/cil/src/cil_lexer.l FAILED: /bin/bash -c "prebuilts/misc/linux-x86/flex/flex-2.5.39 -o/home/laserbeamh/halium/out/host/linux-x86/obj32/SHARED_LIBRARIES/libsepol_intermediates/cil/src/cil_lexer.c external/selinux/libsepol/cil/src/cil_lexer.l" flex-2.5.39: loadlocale.c:130: _nl_intern_locale_data: Assertion cnt < (sizeof (_nl_value_type_LC_TIME) / sizeof (_nl_value_type_LC_TIME[0]))' failed. Aborted (core dumped) [ 7% 1572/22028] host C: libsqlite_32 <= external/sqlite/dist/sqlite3.c ninja: build stopped: subcommand failed. build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed make: *** [ninja_wrapper] Error 1