Issue with Pixel sailfish OPM1 (Android 8.1)

[grimaudjm]

Hello, I face an issue when I use vendor/google_devices extracted from sailfish-opm1.171019.011-factory-56d15350.zip When I build my full image, the phone does not start while:

• it is ok if I use the pre-built vendor image from google
• it was ok with Android 8.0 when I used android-preapare to extract vendor files.

I would like to know if someone already made the test. here is the log of android-prepare for Android 8.1: ./execute-all.sh --device sailfish --buildID OPM1 -i sailfish-opm1.171019.011-factory-56d15350.zip --output $(pwd) [] Setting output base to '/work/android/users/frq02541/Pixel/android-prepare-vendor/sailfish/opm1' [] Extracting '/work/android/users/frq02541/Pixel/android-prepare-vendor/sailfish-opm1.171019.011-factory-56d15350.zip' [] Unzipping 'image-sailfish-opm1.171019.011.zip' [] Processing with 'API-27 naked' configuration [] '3' bytecode archive files will be repaired [] Repairing bytecode under /system partition using oatdump method [!] '/priv-app/qcrilmsgtunnel/qcrilmsgtunnel.apk' not pre-optimized with sanity checks passed - copying without changes [!] '/priv-app/Asdiv/Asdiv.apk' not pre-optimized with sanity checks passed - copying without changes [!] '/priv-app/CNEService/CNEService.apk' not pre-optimized with sanity checks passed - copying without changes cp: cannot open `/work/android/users/frq02541/Pixel/android-prepare-vendor/sailfish/opm1/factory_imgs_data/system/system/etc/fs_config_dirs' for reading: Permission denied [!] Failed to copy '/etc/fs_config_dirs' - skipping [] System partition successfully extracted & repaired at '/work/android/users/frq02541/Pixel/android-prepare-vendor/sailfish/opm1/factory_imgs_repaired_data' [] Generating 'sailfish' vendor blobs [] Copying radio files '/work/android/users/frq02541/Pixel/android-prepare-vendor/sailfish/opm1/vendor/google_devices/sailfish' [] Copying product files & generating 'sailfish-vendor-blobs.mk' makefile [] Generating 'device-vendor-sailfish.mk' [] Generating 'AndroidBoardVendor.mk' [] Bootloader:8996-012001-1710040120 [] Baseband:8996-130091-1710201747 [] Generating 'BoardConfigVendor.mk' [] Generating 'vendor-board-info.txt' [] Generating 'Android.mk' [] Gathering data from 'vendor/app' APK/JAR pre-builts [] Gathering data from 'vendor/framework' APK/JAR pre-builts [] Gathering data from 'vendor/overlay/Pixel' APK/JAR pre-builts [] Gathering data from 'proprietary/priv-app' APK/JAR pre-builts [!] Orphan bytecode file detected 'framework-res__auto_generated_rro.apk' & removed [!] Orphan bytecode file detected 'SysuiDarkThemeOverlay.apk' & removed [] Processing standalone symlinks [] Generating signatures file [] Generating build_id file [] Import '/work/android/users/frq02541/Pixel/android-prepare-vendor/sailfish/opm1/vendor' vendor blobs to AOSP root [] Import '/work/android/users/frq02541/Pixel/android-prepare-vendor/sailfish/opm1/vendor_overlay' vendor overlays to AOSP root [*] All actions completed successfully

[anestisb]

Thanks for reporting. I'll investigate it. Have you made a clean build before testing?

[grimaudjm]

Hello, thank you for your answer. yes I have made a clean build.

[anestisb]

Ok I've managed to reproduce the issue against a sailfish release build with the naked config blobs extracted from the opm1.171019.011 factory images. It seems that something is missing (or not properly included) from the vendor partition. Will investigate on Friday that I'll have access to my UART debug cables.

It's very weird how this passed original testing and for some reason is not affecting CopperheadOS that uses the generated blobs.

Btw the same setup against a userdebug build instead of a release one is working fine.

[grimaudjm]

Hi anestisb, many thanks for your feedback. Could you please kindly indicate me where I could find a userdebug build ? I downloaded the official release from https://developers.google.com/android/images, and I do not see userdebug images.

[anestisb]

I mean the userdebug BUILDTYPE (the one you pass to lunch) that matches the sailfish device. In this case you need the aosp_sailfish-user target.

[grimaudjm]

ok understood. But when working on AOSP, only choice for sailfish is aosp_sailfish-userdebug, so I already use it and I face the issue.

[anestisb]

From the vanilla AOSP tree I use the aosp_sailfish-user target to build the Pixel release image and the aosp_sailfish-userdebug target to build the debugging image. The userdebug image is working fine with the generated blobs. I can reproduce your issue only when building with the release image and the vendor blobs generated from this tool.

[grimaudjm]

This is interesting, thanks. Here is my build process: repo init -u https://android.googlesource.com/platform/manifest -b android-8.1.0_r1 repo sync ./extract-qcom-sailfish.sh (version qcom-sailfish-opm1.171019.011-247af472.tgz) cp -fr android-prepare-dir/sailfish/opm1/vendor/google_devices ./vendor . build/envsetup.sh lunch aosp_sailfish-userdebug (I do not see the aosp_sailfish-user option) make

Once the build done, I open the official zip image-sailfish-opm1.171019.011.zip and I replace in it the files system.img, system_other.img, vendor.img, userdebug.img and boot.img and execute flash-all.bat script

Then the phone does not boot.

[anestisb]

Ok found the problem. Some debugfs versions are failing to properly handle the symlinks that are present in the factory vendor partition. As such the toybox_vendor links are not picked-up and the utility binary paths are not present in the generated image. This causes some vendor init scripts to fail.

The problem at this point is that not all debugfs versions present this behaviour, thus the mix up of the issues reproducibility. For example the debugfs 1.42.13 (17-May-2015) version under an Ubuntu 16.04.03 box is buggy, although the 1.42.12 (29-Aug-2014) under a Debian jessie is not.

The debugfs rdump command errors are like this:

rdump: Attempt to read block from filesystem resulted in short read while reading symlink

A temporarily solution would be to revert back to fuse-ext2 by applying the following patch:

diff --git a/execute-all.sh b/execute-all.sh
index 1a4cbe0..c6f06ea 100755
--- a/execute-all.sh
+++ b/execute-all.sh
@@ -321,7 +321,9 @@ if isDarwin; then
   _UMOUNT=umount
 else
   # For Linux use debugfs
-  USE_DEBUGFS=true
+  USE_DEBUGFS=false
+  SYS_TOOLS+=("fusermount")
+  _UMOUNT="fusermount -u"
 fi

 # Check that system tools exist

Although, I've disabled fuse-ext2 since it was highly unstable. If I don't find a reliable workaround, I would either have to use a different fuse-based tool, or revert back to initial logic where the script required root access to normally mount the extract partitions.

I'll do some research the next few days and update this post when the issue is properly fixed.

[grimaudjm]

Hi anestisb, I wish you an happy new year ! Thanks for your great support. I made some tests with fuse-ext2, on Ubuntu. unfortunatly I have different issues, probably linked to sudo rights.

I have such kind of error: [-] '/tmp/android_img_extract.MPWAf7/sailfish-opm1.171019.011-factory-56d15350/images/system.img.raw' mount point missing indicates fuse mount error

[grimaudjm]

for more details, I have executed by hand the fuse-ext2 command, and here is the log: fuse-ext2 -o uid=$EUID,ro /tmp/android_img_extract.Df0nBu/sailfish-opm1.171019.011-factory-56d15350/images/system.img.raw /work/android/users/frq02541/Pixel/android-prepare-vendor/sailfish/opm1/factory_imgs_data/system/ fuse-umfuse-ext2: version:'0.4', fuse_version:'29' [main (fuse-ext2.c:331)] fuse-umfuse-ext2: enter [do_probe (do_probe.c:30)] fuse-umfuse-ext2: leave [do_probe (do_probe.c:55)] fuse-umfuse-ext2: opts.device: /tmp/android_img_extract.Df0nBu/sailfish-opm1.171019.011-factory-56d15350/images/system.img.raw [main (fuse-ext2.c:358)] fuse-umfuse-ext2: opts.mnt_point: /work/android/users/frq02541/Pixel/android-prepare-vendor/sailfish/opm1/factory_imgs_data/system/ [main (fuse-ext2.c:359)] fuse-umfuse-ext2: opts.volname: / [main (fuse-ext2.c:360)] fuse-umfuse-ext2: opts.options: uid=5280,ro [main (fuse-ext2.c:361)] fuse-umfuse-ext2: parsed_options: uid=5280,ro,fsname=/tmp/android_img_extract.Df0nBu/sailfish-opm1.171019.011-factory-56d15350/images/system.img.raw [main (fuse-ext2.c:362)] fuse-umfuse-ext2: mounting read-only [main (fuse-ext2.c:378)] fusermount: failed to open mountpoint for reading: Permission denied

[anestisb]

Have you added the user_allow_other option in the /etc/fuse.conf?

[grimaudjm]

no I haven't. Thank you for your support, I will check.

[grimaudjm]

hello, sorry for the late answer. I have been able to modify the file /etc/fuse.conf (uncomment user_allow_other) but I still have the same issue.

[grimaudjm]

Hello, here are some news: I have been able to execute fuse-ext2 by adding the following line juste before calling fuse-ext2 into extract-factory-images.sh chmod 777 "$mountPoint"

Now I have an issue with simg2img

./execute-all.sh --device sailfish --buildID OPM1 -i sailfish-opm1.171019.012-factory-49ed1aec.zip --output $(pwd)

[] Setting output base to '/work/android/users/frq02541/Pixel/android-prepare-vendor/sailfish/opm1' [] Extracting '/work/android/users/frq02541/Pixel/android-prepare-vendor/sailfish-opm1.171019.012-factory-49ed1aec.zip' [*] Unzipping 'image-sailfish-opm1.171019.012.zip' error: file_write: incomplete write Cannot write output file [-] simg2img failed to convert system.img from sparse [-] Factory images data extract failed

Any idea ? Again thank a lot for your support.

[anestisb]

I'm not sure why simg2img is failing, it seems that the actual write syscall is failing. Does your output directory have an fs other than ext4?

So far I haven't found any fuse solution that can work decently for most Linux hosts. So I'm thinking of putting back the option to run the script as root and mount directly the converted sparse images.

[grimaudjm]

Hello, I think I finally have something working on Android 8.1 ! Forget my last message concerning simg2img, I think it was my mistake.

Here are the modifications I need to apply, considering that I have no root access on my Ubuntu station. It concerns fuse-ext2 execution: I need to modify the rights of $mountPoint before calling fuse-ext2, and I need to use option rw+ instead of ro (read only).

chmod 777 -R "$mountPoint" fuse-ext2 -o uid=$EUID,rw+ "$imgFile" "$mountPoint" &>"$mount_log" ||

I you have any comments, they are welcome !

[anestisb]

The "rw+" part if very weird.

Unfortunately, I'm really busy these days so no time left to clear this up. Hopefully will find some time within February to do some more testing and apply a proper fix for the fuses, or worst case scenario some more targeted instructions on how to prepare the fuse env.

Thanks for your comments. Will leave the issue open for the time being until I can decide on a proper patch-set for the upstream.

[grimaudjm]

Hello, could you please kindly tell me if you have any update ?

[grimaudjm]

I have an update on myside. I have an issue when I flash Android 8.1 vendor.img built from output of android-prepare. I found that it comes from some makefile : Android.mk and device-vendor-sailfish.mk I had to apply the following command to clean them: find . -name "*.mk" -exec sed -i -r 's/__[0-9]+//' {} \;

[anestisb]

@stnfc unfortunately I didn't find the time yet to clear-up the ext4-fuses issues.

Regarding your latest issue that you had to replace the __[0-9]+ symbolic link targets in the makefiles what exactly was your error in your build infrastructure? Can you please copy some indicative error logs from you build env?

Both me and copperheadOS (and other rom developers) are using the tool's output without any issue when compiling from the upstream AOSP.

[grimaudjm]

thanks for your answer. In fact without this trick, I did not see any symbolic link in the directory $OUT/vendor/bin There was not any error during the build, but the phone did not boot. It was blocked in Android screen. I implemented this trick because it was like this with Android 8.0

[anestisb]

@stnfc I've replaced the buggy fuse-ext2 with ext4fuse (you can compile from sources for Linux). I've also disabled the debugfs use by default on Linux since it appears that the Ubuntu version has issues with the symbolic links.

It should be ok now. Let me know if you still face issues in your setup. If so, I'll add a third alternative for Linux without fuses that will require root privileges on the corresponding script.