search

anestisb / android-prepare-vendor

Set of scripts to automate AOSP compatible vendor blobs generation from factory images
347 stars 156 forks source link

blueline (Pixel 3) support #136

Closed dan-v closed 5 years ago

dan-v commented 5 years ago

I don't have a device yet, so haven't played around with building an bootable image yet: https://developers.google.com/android/images#blueline. Figured it would be good to have an open item to track any progress though.

cernekee commented 5 years ago

Here's my first crack at it: https://github.com/cernekee/android-prepare-vendor/commit/a0cf940eec20ca754980c3b53c48ebe0c4044fbe

Unfortunately, the image doesn't boot. Basically what I tried so far was:

0) Fix ext4fuse so it can read the Nexus image: https://github.com/gerard/ext4fuse/pull/51

1) Update the extra-partitions list based on the delta between walleye and blueline Nexus images

2) For each file under proprietary/ in the official self-extracting blueline vendor blobs, find the path in the Nexus image and list it under system-bytecode or system-other

3) Look at the list of walleye APKs, and for each one that exists on blueline too, add that to the list

There were 9 extra files, mostly "radioconfig" stuff, which appeared in the self-extracting vendor blobs but didn't appear in the Nexus image. Not quite sure what to make of that. But I'm guessing that @anestisb has a better process for figuring out which files are needed.

Also, I get a bunch of warnings during the build, so something isn't right:

[969/969] including vendor/google_devices/blueline/Android.mk ...
build/make/core/Makefile:28: warning: overriding commands for target `out/target/product/blueline/vendor/etc/IPACM_cfg.xml'
build/make/core/base_rules.mk:412: warning: ignoring old commands for target `out/target/product/blueline/vendor/etc/IPACM_cfg.xml'
build/make/core/Makefile:28: warning: overriding commands for target `out/target/product/blueline/system/lib64/liblogwrap.so'
build/make/core/base_rules.mk:412: warning: ignoring old commands for target `out/target/product/blueline/system/lib64/liblogwrap.so'
build/make/core/Makefile:28: warning: overriding commands for target `out/target/product/blueline/system/lib64/libprotobuf-cpp-full-rtti.so'
build/make/core/base_rules.mk:412: warning: ignoring old commands for target `out/target/product/blueline/system/lib64/libprotobuf-cpp-full-rtti.so'
build/make/core/Makefile:28: warning: overriding commands for target `out/target/product/blueline/system/lib64/libprotobuf-cpp-full.so'
build/make/core/base_rules.mk:412: warning: ignoring old commands for target `out/target/product/blueline/system/lib64/libprotobuf-cpp-full.so'
build/make/core/Makefile:28: warning: overriding commands for target `out/target/product/blueline/system/lib64/vndk-28/libprotobuf-cpp-full.so'
build/make/core/base_rules.mk:412: warning: ignoring old commands for target `out/target/product/blueline/system/lib64/vndk-28/libprotobuf-cpp-full.so'
build/make/core/Makefile:28: warning: overriding commands for target `out/target/product/blueline/system/lib/liblogwrap.so'
build/make/core/base_rules.mk:412: warning: ignoring old commands for target `out/target/product/blueline/system/lib/liblogwrap.so'
build/make/core/Makefile:28: warning: overriding commands for target `out/target/product/blueline/system/lib/libprotobuf-cpp-full-rtti.so'
build/make/core/base_rules.mk:412: warning: ignoring old commands for target `out/target/product/blueline/system/lib/libprotobuf-cpp-full-rtti.so'
build/make/core/Makefile:28: warning: overriding commands for target `out/target/product/blueline/system/lib/libprotobuf-cpp-full.so'
build/make/core/base_rules.mk:412: warning: ignoring old commands for target `out/target/product/blueline/system/lib/libprotobuf-cpp-full.so'
build/make/core/Makefile:28: warning: overriding commands for target `out/target/product/blueline/system/lib/vndk-28/libprotobuf-cpp-full.so'
build/make/core/base_rules.mk:412: warning: ignoring old commands for target `out/target/product/blueline/system/lib/vndk-28/libprotobuf-cpp-full.so'
build/make/core/Makefile:28: warning: overriding commands for target `out/target/product/blueline/vendor/bin/awk'
build/make/core/base_rules.mk:412: warning: ignoring old commands for target `out/target/product/blueline/vendor/bin/awk'
build/make/core/Makefile:28: warning: overriding commands for target `out/target/product/blueline/vendor/bin/grep'
build/make/core/base_rules.mk:412: warning: ignoring old commands for target `out/target/product/blueline/vendor/bin/grep'
build/make/core/Makefile:28: warning: overriding commands for target `out/target/product/blueline/vendor/bin/hostapd_cli'
[...]
build/make/core/Makefile:28: warning: overriding commands for target `out/target/product/blueline/vendor/lib/soundfx/libbundlewrapper.so'
build/make/core/base_rules.mk:412: warning: ignoring old commands for target `out/target/product/blueline/vendor/lib/soundfx/libbundlewrapper.so'
build/make/core/Makefile:28: warning: overriding commands for target `out/target/product/blueline/vendor/lib/soundfx/libdownmix.so'
build/make/core/base_rules.mk:412: warning: ignoring old commands for target `out/target/product/blueline/vendor/lib/soundfx/libdownmix.so'
build/make/core/Makefile:28: warning: overriding commands for target `out/target/product/blueline/vendor/lib/soundfx/libdynproc.so'
build/make/core/base_rules.mk:412: warning: ignoring old commands for target `out/target/product/blueline/vendor/lib/soundfx/libdynproc.so'
build/make/core/Makefile:28: warning: overriding commands for target `out/target/product/blueline/vendor/lib/soundfx/libeffectproxy.so'
build/make/core/base_rules.mk:412: warning: ignoring old commands for target `out/target/product/blueline/vendor/lib/soundfx/libeffectproxy.so'
build/make/core/Makefile:28: warning: overriding commands for target `out/target/product/blueline/vendor/lib/soundfx/libldnhncr.so'
build/make/core/base_rules.mk:412: warning: ignoring old commands for target `out/target/product/blueline/vendor/lib/soundfx/libldnhncr.so'
build/make/core/Makefile:28: warning: overriding commands for target `out/target/product/blueline/vendor/lib/soundfx/libmalistener.so'
build/make/core/base_rules.mk:412: warning: ignoring old commands for target `out/target/product/blueline/vendor/lib/soundfx/libmalistener.so'
build/make/core/Makefile:28: warning: overriding commands for target `out/target/product/blueline/vendor/lib/soundfx/libqcompostprocbundle.so'
build/make/core/base_rules.mk:412: warning: ignoring old commands for target `out/target/product/blueline/vendor/lib/soundfx/libqcompostprocbundle.so'
build/make/core/Makefile:28: warning: overriding commands for target `out/target/product/blueline/vendor/lib/soundfx/libqcomvisualizer.so'
build/make/core/base_rules.mk:412: warning: ignoring old commands for target `out/target/product/blueline/vendor/lib/soundfx/libqcomvisualizer.so'
build/make/core/Makefile:28: warning: overriding commands for target `out/target/product/blueline/vendor/lib/soundfx/libqcomvoiceprocessingdescriptors.so'
build/make/core/base_rules.mk:412: warning: ignoring old commands for target `out/target/product/blueline/vendor/lib/soundfx/libqcomvoiceprocessingdescriptors.so'
build/make/core/Makefile:28: warning: overriding commands for target `out/target/product/blueline/vendor/lib/soundfx/libqcomvoiceprocessing.so'
build/make/core/base_rules.mk:412: warning: ignoring old commands for target `out/target/product/blueline/vendor/lib/soundfx/libqcomvoiceprocessing.so'
build/make/core/Makefile:28: warning: overriding commands for target `out/target/product/blueline/vendor/lib/soundfx/libreverbwrapper.so'
build/make/core/base_rules.mk:412: warning: ignoring old commands for target `out/target/product/blueline/vendor/lib/soundfx/libreverbwrapper.so'
build/make/core/Makefile:28: warning: overriding commands for target `out/target/product/blueline/vendor/lib/soundfx/libvisualizer.so'
build/make/core/base_rules.mk:412: warning: ignoring old commands for target `out/target/product/blueline/vendor/lib/soundfx/libvisualizer.so'
build/make/core/Makefile:28: warning: overriding commands for target `out/target/product/blueline/vendor/lib/soundfx/libvolumelistener.so'
build/make/core/base_rules.mk:412: warning: ignoring old commands for target `out/target/product/blueline/vendor/lib/soundfx/libvolumelistener.so'
build/make/core/Makefile:2072: warning: overriding commands for target `out/target/product/blueline/product.img'
vendor/google_devices/blueline/AndroidBoardVendor.mk:15: warning: ignoring old commands for target `out/target/product/blueline/product.img'
cernekee commented 5 years ago

Latest attempt here: https://github.com/cernekee/android-prepare-vendor/commit/a9995444a0a9f16f2bd8cd65ec066e1e5d2d7c0f

This is probably "good enough for now" unless someone has additional suggestions.

I compared a system image generated using a-p-v with a system image generated using the self-extracting blobs. a-p-v adds a couple of extra files; not sure if they're useful (or harmful). The a-p-v version is missing these files:

system/etc/init/init-persist.rc
system/etc/permissions/radioconfiginterface.xml
system/etc/permissions/radioconfig.xml
system/etc/permissions/uimremoteclient.xml
system/etc/permissions/uimremoteserver.xml
system/framework/LteDirectDiscoveryLibrary.jar
system/framework/radioconfiginterfacelibrary.jar
system/framework/radioconfiglibrary.jar
system/framework/uimremoteclientlibrary.jar
system/framework/uimremoteserverlibrary.jar
system/lib64/libDiagService.so

Mostly looks like Java libraries + corresponding permission files. The Nexus image had the native components but not the JARs.

Comparing the vendor partitions between the Nexus image and the self-extracting blobs, it doesn't look like we're missing anything. A whole bunch of files had different hashes. It wasn't clear to me whether these were materially different versions of the libraries, or just slightly different builds.

The a-p-v blueline image seems to mostly work as well as the version built using the self-extracting blobs. Couple things I noticed:

1) Phone audio doesn't work on either build :-( https://issuetracker.google.com/issues/118921806

2) The self-extracting blob enables the new home button interface. The a-p-v build enables the old UI. I assume there's a property somewhere that is getting flipped.

3) The default wallpaper is also different between the two builds.

Also, I get a bunch of warnings during the build

FWIW I saw some of these (albeit a smaller number) using the official self-extracting blobs.

Also, I proposed a patch to fix signing AOSP images built with the self-extracting vendor blobs:

https://android-review.googlesource.com/813295

anestisb commented 5 years ago

@cernekee, @dan-v appreciate your contributions. Will start processing them and the general Pixel 3 support this week.

CaseyBakey commented 5 years ago

Also, I get a bunch of warnings during the build, so something isn't right:

[969/969] including vendor/google_devices/blueline/Android.mk ...
build/make/core/base_rules.mk:412: warning: ignoring old commands for target `out/target/product/blueline/vendor/lib/soundfx/libvisualizer.so'
build/make/core/Makefile:28: warning: overriding commands for target `out/target/product/blueline/vendor/lib/soundfx/libvolumelistener.so'
build/make/core/base_rules.mk:412: warning: ignoring old commands for target `out/target/product/blueline/vendor/lib/soundfx/libvolumelistener.so'
build/make/core/Makefile:2072: warning: overriding commands for target `out/target/product/blueline/product.img'
vendor/google_devices/blueline/AndroidBoardVendor.mk:15: warning: ignoring old commands for target `out/target/product/blueline/product.img'

Did you manage to understand why you did get these warnings? If yes, how did you fix them?

thestinger commented 5 years ago

It generally means that there are duplicate rules, i.e. it's being built / installed by at least two different parts of the build system. A whole lot of the vendor partition can actually be built from AOSP and there are many rules trying to do exactly that. There are a lot more that could be used, but aren't trying to build parts of it without putting in some effort.

There are also large amounts of it with sources published in the Code Aurora Forum repositories, which are presumably part of Google's internal vendor repository. Google just isn't putting in the effort to try to make as much of it public as they can. Instead, they just expect AOSP users to use the prebuilt vendor.img and live without full functionality like verified boot and a matching fingerprint to avoid a warning. It's a bit silly. They should put in some more effort and include building vendor.img in AOSP along with publishing as much as they can to be built from source in AOSP, with blobs + a build system for them provided for the rest, which is much less than it seems at first glance.

CaseyBakey commented 5 years ago

Thanks for your answer @thestinger.

I was also thinking of duplicate build targets, but I don't get why it would mess my final build.

Blueline, 9.0.0_r34 here. BL unlocked, using "full" configuration. I get a red screen with an unbootable system. With "naked" configuration, orange screen then yellow if I relock.

Do you know why it could happen, and how should I track down the duplicate targets (to prefer the android-prepare-vendor ones)?

I'm sorry if I kinda hijacked this "closed" issue, but it's exactly the same behavior as @cernekee I'm getting.

Thanks

thestinger commented 5 years ago

I don't know. I recommend getting a debug cable to debug early boot problems.

https://www.sparkfun.com/products/14746

CaseyBakey commented 5 years ago

Can you confirm this cable will work on Pixel 3/XL?

By the way, I think this issue should be reopened, at least for the "full" config. "naked" config seems to be working fine (at least the phone boot).

thestinger commented 5 years ago

Yes, it supports the Pixel 2 / 2 XL / 3 / 3 XL.

anestisb commented 5 years ago

Can you confirm this cable will work on Pixel 3/XL?

By the way, I think this issue should be reopened, at least for the "full" config. "naked" config seems to be working fine (at least the phone boot).

To be honest I've performed limited testing with full config since OpenGapps wasn't available at that point and didn't test more. Please open a new issue with all the information you managed to get so far about the bug and how to reproduce it, and I'll try to have a look.