lineageos4microg / docker-lineage-cicd

Docker microservice for LineageOS Continuous Integration and Continous Deployment
https://hub.docker.com/r/lineageos4microg/docker-lineage-cicd
GNU General Public License v3.0
480 stars 189 forks source link

Restore A14 patches (not working), fix user patch #615

Closed FintasticMan closed 2 months ago

FintasticMan commented 2 months ago

This restores the (still currently non-functional) Android 14 signature spoofing patches, so that, once they have been fixed, users will be able to create unrestricted patches.

These changes also mean that the error faced by @gwstorm is fixed.

If these are merged, the nns-v21-spoofing-patches branch no longer has any loss of functionality compared to master, and only improves compatibility. Therefore I believe that it should be able to be merged into master.

After (or before, but then there could be merge conflicts) that, we can focus on fixing the patches so that they work. I see you've already started work on that, thank you!

petefoth commented 2 months ago

Thanks for this - merged

gwstorm commented 2 months ago

These changes also mean that the error faced by @gwstorm is fixed.

Thanks for all of this work, guys. Indeed, I got past the build script error. The build is failing later on, but I don't think it has anything to do with the any of the work being done here.

FintasticMan commented 2 months ago

For the moment, you need to disable the SIGNATURE_SPOOFING option (and I would recommend disabling it in future as well).

gwstorm commented 2 months ago

Understood. I'll keep it disabled. Same error though,

Error ``` ... INSTALL techpack/datarmnet/core/rmnet_core.ko INSTALL techpack/datarmnet/core/rmnet_ctl.ko INSTALL techpack/display/msm/msm_drm.ko DEPMOD 5.4.268-qgki-g803bbce1eee5 make[1]: Leaving directory '/srv/src/LINEAGE_21_0/out/target/product/lemonadep/obj/KERNEL_OBJ' make: Leaving directory '/srv/src/LINEAGE_21_0/kernel/oneplus/sm8350' depmod: WARNING: could not open modules.order at /srv/src/LINEAGE_21_0/out/target/product/lemonadep/obj/PACKAGING/depmod_vendor_intermediat> depmod: WARNING: could not open modules.builtin at /srv/src/LINEAGE_21_0/out/target/product/lemonadep/obj/PACKAGING/depmod_vendor_intermedi> depmod: WARNING: could not open modules.builtin.modinfo at /srv/src/LINEAGE_21_0/out/target/product/lemonadep/obj/PACKAGING/depmod_vendor_i> depmod: WARNING: could not open modules.order at /srv/src/LINEAGE_21_0/out/target/product/lemonadep/obj/PACKAGING/depmod_vendor_ramdisk_int> depmod: WARNING: could not open modules.builtin at /srv/src/LINEAGE_21_0/out/target/product/lemonadep/obj/PACKAGING/depmod_vendor_ramdisk_i> depmod: WARNING: could not open modules.builtin.modinfo at /srv/src/LINEAGE_21_0/out/target/product/lemonadep/obj/PACKAGING/depmod_vendor_r> ninja: build stopped: subcommand failed. 23:28:24 ninja failed with: exit status 1 #### failed to build some targets (01:04:26 (hh:mm:ss)) #### >> [Wed Apr 24 23:28:24 UTC 2024] Failed build for lemonadep >> [Wed Apr 24 23:28:25 UTC 2024] Finishing build for lemonadep >> [Wed Apr 24 23:28:25 UTC 2024] Cleaning up >> [Wed Apr 24 23:28:25 UTC 2024] Cleaning source dir for device lemonadep 23:28:34 Build sandboxing disabled due to nsjail error. 23:29:33 Entire build directory removed. #### build completed successfully (01:08 (mm:ss)) #### ```

Assuming it's still unrelated.

petefoth commented 2 months ago

Understood. I'll keep it disabled. Same error though, Error

Assuming it's still unrelated.

The part of the log files you pasted does not show where the error occurred or when it occurred. To find that out, you need to search in the log file for the string FAILED:. The line before that is the command that failed, and subsequent lines, down to the next % progress line give details of the error. That's what you need to look at - or post here - to find the cause of the error

There will be a lot of text in the log file after the error occurs. This is because there are several build tasks running concurrently in different threads and the build won't stop until all those threads complete. One of the threads is probably Building Kernel Image. This task produces lots of log output which is not written to the log until the thread completes, usually some time after the FAILED command. So the stuff in your paste from the log, from INSTALL techpack/datarmnet/core/rmnet_core.ko down to >> [Wed Apr 24 23:28:25 UTC 2024] Cleaning source dir for device lemonadep

Is output from the Building Kernel Image task and not from the failed command

gwstorm commented 2 months ago

Got it. Definitely learning a lot about this now. If this is off topic and better suited for its own issue, let me know.

Here's the actual failure point ``` [ 98% 181644/184025] Target system_ext fs image: out/target/product/lemonadep/system_ext.img (priority: 8) FAILED: out/target/product/lemonadep/system_ext.img /bin/bash -c "(mkdir -p out/target/product/lemonadep/system_ext ) && (mkdir -p out/target/product/lemonadep/obj/PACKAGING/system_ext_intermediates && rm -rf out/target/product/lemonadep/obj/PACKAGING/system_ext_intermediates/system_ext_image_info.txt ) && (echo \"system_ext_fs_type=ext4\" >> out/target/product/lemonadep/obj/PACKAGING/system_ext_intermediates/system_ext_image_info.txt ) && (echo \"system_ext_reserved_size=8388609\" >> out/target/product/lemonadep/obj/PACKAGING/system_ext_intermediates/system_ext_image_info.txt ) && (echo \"system_ext_disable_sparse=true\" >> out/target/product/lemonadep/obj/PACKAGING/system_ext_intermediates/system_ext_image_info.txt ) && (echo \"system_ext_selinux_fc=out/target/product/lemonadep/obj/ETC/file_contexts.bin_intermediates/file_contexts.bin\" >> out/target/product/lemonadep/obj/PACKAGING/system_ext_intermediates/system_ext_image_info.txt ) && (echo \"building_system_ext_image=true\" >> out/target/product/lemonadep/obj/PACKAGING/system_ext_intermediates/system_ext_image_info.txt ) && (echo \"ext_mkuserimg=mkuserimg_mke2fs\" >> out/target/product/lemonadep/obj/PACKAGING/system_ext_intermediates/system_ext_image_info.txt ) && (echo \"fs_type=ext4\" >> out/target/product/lemonadep/obj/PACKAGING/system_ext_intermediates/system_ext_image_info.txt ) && (echo \"extfs_sparse_flag=-s\" >> out/target/product/lemonadep/obj/PACKAGING/system_ext_intermediates/system_ext_image_info.txt ) && (echo \"erofs_sparse_flag=-s\" >> out/target/product/lemonadep/obj/PACKAGING/system_ext_intermediates/system_ext_image_info.txt ) && (echo \"squashfs_sparse_flag=-s\" >> out/target/product/lemonadep/obj/PACKAGING/system_ext_intermediates/system_ext_image_info.txt ) && (echo \"f2fs_sparse_flag=-S\" >> out/target/product/lemonadep/obj/PACKAGING/system_ext_intermediates/system_ext_image_info.txt ) && (echo \"erofs_default_compressor=\"lz4hc,9\"\" >> out/target/product/lemonadep/obj/PACKAGING/system_ext_intermediates/system_ext_image_info.txt ) && (echo \"avb_avbtool=avbtool\" >> out/target/product/lemonadep/obj/PACKAGING/system_ext_intermediates/system_ext_image_info.txt ) && (echo \"avb_system_ext_hashtree_enable=true\" >> out/target/product/lemonadep/obj/PACKAGING/system_ext_intermediates/system_ext_image_info.txt ) && (echo \"avb_system_ext_add_hashtree_footer_args=--prop com.android.build.system_ext.os_version:14 --prop com.android.build.system_ext.fingerprint:\$(cat out/target/product/lemonadep/build_fingerprint.txt) --prop com.android.build.system_ext.security_patch:2024-04-05\" >> out/target/product/lemonadep/obj/PACKAGING/system_ext_intermediates/system_ext_image_info.txt ) && (echo \"root_dir=out/target/product/lemonadep/root\" >> out/target/product/lemonadep/obj/PACKAGING/system_ext_intermediates/system_ext_image_info.txt ) && (echo \"use_dynamic_partition_size=true\" >> out/target/product/lemonadep/obj/PACKAGING/system_ext_intermediates/system_ext_image_info.txt ) && (echo \"skip_fsck=true\" >> out/target/product/lemonadep/obj/PACKAGING/system_ext_intermediates/system_ext_image_info.txt ) && (sort -o out/target/product/lemonadep/obj/PACKAGING/system_ext_intermediates/system_ext_image_info.txt out/target/product/lemonadep/obj/PACKAGING/system_ext_intermediates/system_ext_image_info.txt ) && (PATH=out/host/linux-x86/bin/:system/extras/ext4_utils/:\$PATH out/host/linux-x86/bin/build_image out/target/product/lemonadep/system_ext out/target/product/lemonadep/obj/PACKAGING/system_ext_intermediates/system_ext_image_info.txt out/target/product/lemonadep/system_ext.img out/target/product/lemonadep/system ) && (true )" 2024-04-24 23:25:07 - build_image.py - ERROR : Failed to build out/target/product/lemonadep/system_ext.img from out/target/product/lemonadep/system_ext Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "/srv/src/LINEAGE_21_0/out/host/linux-x86/bin/build_image/__main__.py", line 12, in File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "build_image.py", line 1038, in File "build_image.py", line 1033, in main File "build_image.py", line 937, in BuildImageOrVBMeta File "build_image.py", line 581, in BuildImage File "build_image.py", line 447, in BuildImageMkfs File "common.py", line 328, in RunAndCheckOutput common.ExternalError: Failed to run command '['mkuserimg_mke2fs', 'out/target/product/lemonadep/system_ext', 'out/target/product/lemonadep/system_ext.img', 'ext4', 'system_ext', '406761472', '-j', '0', '-D', 'out/target/product/lemonadep/system', '-L', 'system_ext', '-i', '3666', '-M', '0', '-U', '12be0001-b2c4-5242-85c9-2e3f32645118', '-S', '5959b069-7333-5a49-9458-c4fa53731533', '--inode_size', '256', 'out/target/product/lemonadep/obj/ETC/file_contexts.bin_intermediates/file_contexts.bin']' (exit code 4): 23:25:06 mkuserimg_mke2fs.py INFO: Env: {'MKE2FS_CONFIG': '/srv/src/LINEAGE_21_0/out/soong/.temp/tmp2zat1x3t'} 23:25:06 mkuserimg_mke2fs.py INFO: Running: /srv/src/LINEAGE_21_0/out/host/linux-x86/bin/mke2fs -O ^has_journal -L system_ext -N 3666 -I 256 -M /system_ext -m 0 -U 12be0001-b2c4-5242-85c9-2e3f32645118 -E hash_seed=5959b069-7333-5a49-9458-c4fa53731533 -t ext4 -b 4096 out/target/product/lemonadep/system_ext.img 99307 23:25:07 mkuserimg_mke2fs.py INFO: Env: {} 23:25:07 mkuserimg_mke2fs.py INFO: Running: /srv/src/LINEAGE_21_0/out/host/linux-x86/bin/e2fsdroid -e -p out/target/product/lemonadep/system -S out/target/product/lemonadep/obj/ETC/file_contexts.bin_intermediates/file_contexts.bin -f out/target/product/lemonadep/system_ext -a /system_ext out/target/product/lemonadep/system_ext.img 23:25:07 mkuserimg_mke2fs.py ERROR: Failed to run e2fsdroid_cmd: __populate_fs: Could not allocate block in ext2 filesystem while writing file "spup.vim" e2fsdroid: Could not allocate block in ext2 filesystem while populating file system mke2fs 1.46.6 (1-Feb-2023) Creating filesystem with 99307 4k blocks and 3712 inodes Filesystem UUID: 12be0001-b2c4-5242-85c9-2e3f32645118 Superblock backups stored on blocks: 32768, 98304 Allocating group tables: 0/4 done Writing inode tables: 0/4 done Writing superblocks and filesystem accounting information: 0/4 done __populate_fs: Could not allocate block in ext2 filesystem while writing file "spup.vim" e2fsdroid: Could not allocate block in ext2 filesystem while populating file system Out of space? Out of inodes? The tree size of out/target/product/lemonadep/system_ext is 398370816 bytes (379 MB), with reserved space of 8388609 bytes (8 MB). The max image size for filesystem files is 406761472 bytes (387 MB), out of a total partition size of 406761472 bytes (387 MB). [ 98% 181645/184025] compile out/host/linux-x86/bin/go/androidmk-lib/pkg/android/soong/androidmk/androidmk.a (priority: 8) ```
FintasticMan commented 2 months ago

Looks like your drive isn't big enough. Might be something else as well though.

petefoth commented 2 months ago

Looks like your drive isn't big enough. Might be something else as well though.

It's nothing to do with the machine being used to build, it's to do with the space available in the syetem_ext partition on the device being built for.

Please can we continue discussion in #616 - this problem is nothing to do with signature spoofing - it occurs when "SIGNATURE_SPOOFING=no" and when building with the master branch (without any of the recent PRs)