For the latest Android devices (Nexus and Pixel), Google is no longer providing
vendor binary archives to be included into AOSP build tree. Officially it is
claimed that all vendor proprietary blobs have been moved to /vendor
partition, which allegedly doesn't need building from users. Unfortunately, that
is not the case since quite a few proprietary executables, DSOs and APKs/JARs
located under /system are required in order to have a fully functional set of
images, although are missing from AOSP public tree. Additionally, if
vendor.img is not generated when system.img is prepared for build, a few
bits are broken that also require manual fixing (various symbolic links between
two partitions, bytecode product packages, vendor shared library dependencies,
etc.).
Everyone's hope is that Google will revise this policy for its devices.
However until then, missing blobs need to be manually extracted from factory
images, processed and included into AOSP tree. These processing steps are
evolving into a total nightmare considering that all recent factory images have
their bytecode (APKs, JARs) pre-optimized to reduce boot time and their original
classes.dex stripped to reduce disk size. As such, these missing prebuilt
components need to be repaired/de-optimized prior to be included, since AOSP
build is not capable to import pre-optimized bytecode modules as part of the
makefile tree.
Scripts & tools included in this repository aim to automate the extraction,
processing and generation of vendor specific data using factory images as
input. Data from vendor partition is mirrored to blob includes via a compatible
makefile structure, so that vendor.img can be generated from AOSP builds while
specially annotating the vendor APKs to maintain pre-signed certificates and not
pre-optimize. If you have modified the build process (such as CyanogenMod) you
might need to apply additional changes in device configurations / makefiles.
The main concept of this tool-set is to apply all required changes in vendor makefiles leaving the AOSP source code tree & build chain untouched. Hacks in AOSP tree, such as those applied by CyanogenMod, are painful to maintain and very fragile.
Repository data is LICENSE free, use it as you want at your own risk. Feedback & patches are more than welcome though.
As of 7.1 release Google has started publishing again a set of vendor blobs for supported Nexus & Pixel devices. Unfortunately the distributed blobs still miss some functionality when compiled under AOSP:
As of Oreo release (8.0) Google has improved the state of the proprietary vendor blobs for Pixel devices. State of supported Nexus devices has not changed much. For Pixel devices most vendor specific resources have been moved to vendor partition and thus simplify & reduce the amount of work that needs to be done to include /system dependencies. Furthermore, the original bytecode is no longer stripped from the factory APKs, enabling an easier inclusion of these resources to the generated vendor makefiles.
The process to extract and import vendor proprietary blobs requires to:
scripts/download-nexus-image.sh)
scripts/extract-factory-images.sh)
vendor.imgscripts/system-img-repair.sh) using one of supported bytecode de-optimization
methods (see next paragraph for details)scripts/generate-vendor.sh)
execute-all.sh runs all previous steps with required order. As an alternative
to download images from Google's website, script can also read factory images
from file-system location using the -i|--img flag.
-k|--keep flag can be used if you want to keep extracted intermediate files
for further investigation. Keep in mind that if used the mount-points from
ext4fuse are not unmounted. So be sure that you manually remove them (or run
the script again without the flag) when done.
All scripts can be executed from macOS, Linux & other Unix-based systems as long as bash 4.x and other utilized command line tools are installed. Scripts will abort if any of the required tools is missing from the host.
Scripts include individual usage info and additional flags that be used for targeted advanced actions, bugs investigation & development of new features.
--oatdump flag)Use oatdump host tool (platform/art project from AOSP) to extract DEX
bytecode from OAT's ELF .rodata section. Extracted DEX is not identical to
original since DEX-to-DEX compiler transformations have already been applied
when code was pre-optimized (more info
here).
dexrepair is also used to repair the
extracted DEX file CRC checksum prior to appending bytecode back to matching
APK package from which it has been originally stripped. More info about this
method here.
--smali flag)Use baksmali disassembler against target OAT file to generate a smali syntaxed output. Disassembling process relies on boot framework files (which are automatically include) to resolve class dependencies. Baksmali output is then forwarded to smali assembler to generate a functionally equivalent DEX bytecode file.
--smaliex flag)SmaliEx is an automation tool that is using baksmali/smali at the background and is smoothly handling all the required disassembler/assembler iterations and error handling. Unfortunately due to not quickly catching-up with upstream smali & dexlib it has been deprecated for now.
Naked configuration group (enabled by default when using the master script) includes data & module targets required to have a functional device from AOSP without installing non-essential OEM packages. With this setup using Google Play Services / Google Apps will probably not work.
On the other hand the full configuration group (enabled with -f|--full flag
from master script) has additional blobs & module targets which are normally
marked as non-essential, although might be required for some carriers or in case
of GApps being installed (either manually post-boot or included as additional
vendor blobs).
| Device | API 23 | API 24 | API 25 | API 26 | API 27 | API 28 |
|---|---|---|---|---|---|---|
| N5x bullhead | smaliex smali oatdump |
oatdump smali |
oatdump smali |
oatdump | oatdump | N/A |
| N6p angler | smaliex smali oatdump |
oatdump smali |
oatdump smali |
oatdump | oatdump | N/A |
| N9 flounder WiFi (volantis) |
smaliex smali oatdump |
oatdump smali |
oatdump smali |
N/A | N/A | N/A |
| N9 flounder LTE (volantisg) |
smaliex smali oatdump |
oatdump smali |
oatdump smali |
N/A | N/A | N/A |
| Pixel sailfish | N/A | N/A | oatdump smali |
oatdump | oatdump | oatdump |
| Pixel XL marlin | N/A | N/A | oatdump smali |
oatdump | oatdump | oatdump |
| Pixel 2 walleye | N/A | N/A | N/A | oatdump | oatdump | oatdump |
| Pixel 2 XL taimen | N/A | N/A | N/A | oatdump | oatdump | oatdump |
| Pixel 3 blueline | N/A | N/A | N/A | N/A | N/A | oatdump |
| Pixel 3 XL crosshatch | N/A | N/A | N/A | N/A | N/A | oatdump |
| Pixel 3a sargo | N/A | N/A | N/A | N/A | N/A | oatdump |
| Pixel 3a XL bonito | N/A | N/A | N/A | N/A | N/A | oatdump |
Please check existing issues before reporting new ones
If you want to contribute to device configuration files, please test against the target device before any pull request.
--timestamp option)--fuse-ext2)system/app/TimeService.apk to
extract listLOCAL_DEX_PREOPT overrides from vendor
generate scriptLOCAL_MULTILIB 32 or both when 32bit bytecode prebuilts
detected at 64bit devicesalready defined type of errors when AOSP makefiles are
included, you have other vendor makefiles that define the same packages (e.g.
hammerhead vs bullhead for LGE vendor). This issue is due to other vendor
makefiles not wrapping them with ifeq ($(TARGET_DEVICE),<device_model>).
Wrap conflicting makefiles with device matching clauses to resolve the issue.LOCAL_DEXPREOPT := false.
This is because host dex2oat is invoked with more strict flags and results into
aborting when front-end reaches already optimized instructions. You can use
--force-opt flag if you have modified the default host dex2oat bytecode
pre-compilation flags.update-binary from a factory OTA archive since it's missing from
the AOSP tree due to some proprietary LG code.rsync instead of
cp or mv commands to copy the generated directory structure to different
location. Some device configurations (e.g. Pixel/Pixel XL) share some root
directories and might break if cp or mv are invoked with the wrong base
paths.$ ./execute-all.sh -d flounder -a volantis -b NRD91D -o /fast-datavault/nexus-vendor-blobs
[*] Setting output base to '/fast-datavault/nexus-vendor-blobs/flounder/nrd91d'
--{ Google Terms and Conditions
Downloading of the system image and use of the device software is subject to the
Google Terms of Service [1]. By continuing, you agree to the Google Terms of
Service [1] and Privacy Policy [2]. Your downloading of the system image and use
of the device software may also be subject to certain third-party terms of
service, which can be found in Settings > About phone > Legal information, or as
otherwise provided.
[1] https://www.google.com/intl/en/policies/terms/
[2] https://www.google.com/intl/en/policies/privacy/
[?] I have read and agree with the above terms and conditions - ACKNOWLEDGE [y|n]: y
[*] Downloading image from 'https://dl.google.com/dl/android/aosp/volantis-nrd91d-factory-a27db9bc.zip'
--2016-10-05 21:53:17-- https://dl.google.com/dl/android/aosp/volantis-nrd91d-factory-a27db9bc.zip
Resolving dl.google.com (dl.google.com)... 173.194.76.93, 173.194.76.190, 173.194.76.136, ...
Connecting to dl.google.com (dl.google.com)|173.194.76.93|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 793140236 (756M) [application/zip]
Saving to: ‘/fast-datavault/nexus-vendor-blobs/flounder/nrd91d/volantis-nrd91d-factory-a27db9bc.zip’
s/flounder/nrd91d/volantis-nrd91d-factory-a27db9bc.zip 96%[======================================================================================================================> ] 733.49M 1.22MB/s eta 19s ^/fast-datavault/nexus-vendor-blobs/flounder/nrd91d/vol 100%[==========================================================================================================================>] 756.40M 1.19MB/s in 10m 21s
2016-10-05 22:03:39 (1.22 MB/s) - ‘/fast-datavault/nexus-vendor-blobs/flounder/nrd91d/volantis-nrd91d-factory-a27db9bc.zip’ saved [793140236/793140236]
[*] Processing with 'API-24 config-naked' configuration
[*] Extracting '/fast-datavault/nexus-vendor-blobs/flounder/nrd91d/volantis-nrd91d-factory-a27db9bc.zip'
[*] Unzipping 'image-volantis-nrd91d.zip'
[!] No baseband firmware present - skipping
[!] System partition doesn't contain any pre-optimized files - link to original partition
[*] Generating blobs for vendor/htc/flounder
[*] Copying radio files '/fast-datavault/nexus-vendor-blobs/flounder/nrd91d/vendor/htc/flounder'
[*] Copying product files & generating 'flounder-vendor-blobs.mk' makefile
[*] Generating 'device-vendor.mk'
[*] Generating 'AndroidBoardVendor.mk'
[*] Bootloader:3.48.0.0139
[*] Generating 'BoardConfigVendor.mk'
[*] Generating 'vendor-board-info.txt'
[*] Generating 'Android.mk'
[*] Gathering data from 'vendor/app' APK/JAR pre-builts
[*] Generating signatures file
[*] All actions completed successfully
[*] Import '/fast-datavault/nexus-vendor-blobs/flounder/nrd91d/vendor' to AOSP root$ ./execute-all.sh -d bullhead -i /fast-datavault/nexus-vendor-blobs/bullhead/mtc20k/bullhead-mtc20k-factory-4a950470.zip -b mtc20k -o /fast-datavault/nexus-vendor-blobs
[*] Setting output base to '/fast-datavault/nexus-vendor-blobs/bullhead/mtc20k'
[*] Processing with 'API-23 config-naked' configuration
[*] Extracting '/fast-datavault/nexus-vendor-blobs/bullhead/mtc20k/bullhead-mtc20k-factory-4a950470.zip'
[*] Unzipping 'image-bullhead-mtc20k.zip'
[*] '20' bytecode archive files will be repaired
[*] Repairing bytecode under /system partition using oat2dex method
[*] Preparing environment for 'arm' ABI
[*] Preparing environment for 'arm64' ABI
[*] Start processing system partition & de-optimize pre-compiled bytecode
[!] '/framework/cneapiclient.jar' not pre-optimized with sanity checks passed - copying without changes
[!] '/framework/framework-res.apk' not pre-optimized & without 'classes.dex' - copying without changes
[*] '/framework/framework.jar' is multi-dex - adjusting recursive archive adds
[!] '/framework/rcsimssettings.jar' not pre-optimized with sanity checks passed - copying without changes
[!] '/framework/rcsservice.jar' not pre-optimized with sanity checks passed - copying without changes
[*] System partition successfully extracted & repaired at '/fast-datavault/nexus-vendor-blobs/bullhead/mtc20k/factory_imgs_repaired_data'
[*] Generating blobs for vendor/lge/bullhead
[*] Copying radio files '/fast-datavault/nexus-vendor-blobs/bullhead/mtc20k/vendor/lge/bullhead'
[*] Copying product files & generating 'bullhead-vendor-blobs.mk' makefile
[*] Generating 'device-vendor.mk'
[*] Generating 'AndroidBoardVendor.mk'
[*] Bootloader:BHZ10r
[*] Baseband:M8994F-2.6.32.1.13
[*] Generating 'BoardConfigVendor.mk'
[*] Generating 'vendor-board-info.txt'
[*] Generating 'Android.mk'
[*] Gathering data from 'vendor/app' APK/JAR pre-builts
[*] Gathering data from 'proprietary/app' APK/JAR pre-builts
[*] Gathering data from 'proprietary/framework' APK/JAR pre-builts
[*] Gathering data from 'proprietary/priv-app' APK/JAR pre-builts
[*] Generating signatures file
[*] All actions completed successfully
[*] Import '/fast-datavault/nexus-vendor-blobs/bullhead/mtc20k/vendor' to AOSP root