search

anestisb / android-prepare-vendor

Set of scripts to automate AOSP compatible vendor blobs generation from factory images
347 stars 156 forks source link

Crosshatch device testing #144

Open anestisb opened 5 years ago

anestisb commented 5 years ago

Test a clean build against a Pixel 3 XL device after commit https://github.com/anestisb/android-prepare-vendor/commit/8bc91c5987a7a643574086aaef6256c9933730fe (various config areas & missing blobs were modified).

Unfortunately I wasn't able to secure yet a Pixel 3 XL device, so it would be great if some of the tool users can test this.

iMouath commented 5 years ago

I own a Pixel 3 XL so I decided to test your tools and provide you with feedback.

Disclaimer: I've never built AOSP from source before, some issues could be my own fault.

android-prepare-vendor

While using your tools I've not encountered any issues at all. Image: PQ1A.181205.006 Device: crosshatch

[mouath@desktop android-prepare-vendor]$ bash execute-all.sh -d crosshatch -b PQ1A.181205.006 -o out/ -i images/crosshatch-pq1a.181205.006-factory-96b23504.zip
[*] Setting output base to '/home/mouath/dev/android-prepare-vendor/out/crosshatch/pq1a.181205.006'
[*] Using ext4fuse for image mounts
[*] Extracting '/home/mouath/dev/android-prepare-vendor/images/crosshatch-pq1a.181205.006-factory-96b23504.zip'
[*] Unzipping 'image-crosshatch-pq1a.181205.006.zip'
[*] Processing with 'API-28 naked' configuration
[*] '24' bytecode archive files will be repaired
[*] Repairing bytecode under /system partition using oatdump method
[!] '/app/ims/ims.apk' bytecode is not stripped - copying without changes
[!] '/app/embms/embms.apk' bytecode is not stripped - copying without changes
[!] '/app/datastatusnotification/datastatusnotification.apk' bytecode is not stripped - copying without changes
[!] '/app/QAS_DVC_MSP/QAS_DVC_MSP.apk' bytecode is not stripped - copying without changes
[!] '/app/uimlpaservice/uimlpaservice.apk' bytecode is not stripped - copying without changes
[!] '/app/smcinvokepkgmgr/smcinvokepkgmgr.apk' bytecode is not stripped - copying without changes
[!] '/app/uceShimService/uceShimService.apk' bytecode is not stripped - copying without changes
[!] '/app/com.qualcomm.qti.services.secureui/com.qualcomm.qti.services.secureui.apk' bytecode is not stripped - copying without changes
[!] '/app/remotesimlockservice/remotesimlockservice.apk' bytecode is not stripped - copying without changes
[!] '/app/QtiTelephonyService/QtiTelephonyService.apk' bytecode is not stripped - copying without changes
[!] '/priv-app/CNEService/CNEService.apk' bytecode is not stripped - copying without changes
[!] '/priv-app/ModemService/ModemService.apk' bytecode is not stripped - copying without changes
[!] '/priv-app/grilservice/grilservice.apk' bytecode is not stripped - copying without changes
[!] '/priv-app/qcrilmsgtunnel/qcrilmsgtunnel.apk' bytecode is not stripped - copying without changes
[!] '/framework/embmslibrary.jar' bytecode is not stripped - copying without changes
[*] System partition successfully extracted & repaired at '/home/mouath/dev/android-prepare-vendor/out/crosshatch/pq1a.181205.006/factory_imgs_repaired_data'
[*] Generating 'crosshatch' vendor blobs
[*] Copying radio files '/home/mouath/dev/android-prepare-vendor/out/crosshatch/pq1a.181205.006/vendor/google_devices/crosshatch'
[*] Copying product files & generating 'crosshatch-vendor-blobs.mk' makefile
[*] Generating 'device-vendor.mk'
[*] Generating 'AndroidBoardVendor.mk'
  [*] Bootloader:b1c1-0.1-5034669
  [*] Baseband:g845-00023-180917-B-5014671
[*] Generating 'BoardConfigVendor.mk'
[*] Generating 'vendor-board-info.txt'
[*] Generating 'Android.mk'
[*] Gathering data from 'vendor/app' APK/JAR pre-builts
[*] Gathering data from 'vendor/overlay' APK/JAR pre-builts
[*] Gathering data from 'proprietary/app' APK/JAR pre-builts
[*] Gathering data from 'proprietary/framework' APK/JAR pre-builts
[*] Gathering data from 'proprietary/priv-app' APK/JAR pre-builts
[*] Processing standalone symlinks
[*] Processing additional runtime generated product files from APK symlinks
[*] Generating signatures file
[*] Generating build_id file
[*] Import '/home/mouath/dev/android-prepare-vendor/out/crosshatch/pq1a.181205.006/vendor' vendor blobs to AOSP root
[*] Import '/home/mouath/dev/android-prepare-vendor/out/crosshatch/pq1a.181205.006/vendor_overlay' vendor overlays to AOSP root
[*] All actions completed successfully

AOSP Build

copied what the tool generated to the AOSP source root

cp -a ~/dev/android-prepare-vendor/out/crosshatch/pq1a.181205.006/vendor/google_devices/. vendor/google_devices

Build Vars

============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=9
TARGET_PRODUCT=aosp_crosshatch
TARGET_BUILD_VARIANT=userdebug
TARGET_BUILD_TYPE=release
TARGET_ARCH=arm64
TARGET_ARCH_VARIANT=armv8-2a
TARGET_CPU_VARIANT=cortex-a75
TARGET_2ND_ARCH=arm
TARGET_2ND_ARCH_VARIANT=armv8-a
TARGET_2ND_CPU_VARIANT=cortex-a75
HOST_ARCH=x86_64
HOST_2ND_ARCH=x86
HOST_OS=linux
HOST_OS_EXTRA=Linux-4.20.1-1-MANJARO-x86_64-Manjaro-Linux
HOST_CROSS_OS=windows
HOST_CROSS_ARCH=x86
HOST_CROSS_2ND_ARCH=x86_64
HOST_BUILD_TYPE=release
BUILD_ID=PQ1A.181205.006
OUT_DIR=/mnt/android/hello/build
PRODUCT_SOONG_NAMESPACES=device/google/crosshatch/pixelstats device/google/crosshatch/usb device/google/crosshatch/health
hardware/google/av hardware/google/interfaces hardware/qcom/sdm845 vendor/qcom/sdm845
$ cat /mnt/android/hello/build/soong/soong.variables

{
    "Make_suffix": "-aosp_crosshatch",
    "BuildId": "PQ1A.181205.006",
    "BuildNumberFromFile": "$$(cat /mnt/android/hello/build/build_number.txt)",
    "Platform_version_name": "9",
    "Platform_sdk_version": 28,
    "Platform_sdk_codename": "REL",
    "Platform_sdk_final": true,
    "Platform_version_active_codenames": ["REL"],
    "Platform_version_future_codenames": ["REL"],
    "Allow_missing_dependencies": false,
    "Unbundled_build": false,
    "Pdk": false,
    "Debuggable": true,
    "Eng": false,
    "DeviceName": "crosshatch",
    "DeviceArch": "arm64",
    "DeviceArchVariant": "armv8-2a",
    "DeviceCpuVariant": "cortex-a75",
    "DeviceAbi": ["arm64-v8a"],
    "DeviceSecondaryArch": "arm",
    "DeviceSecondaryArchVariant": "armv8-a",
    "DeviceSecondaryCpuVariant": "cortex-a75",
    "DeviceSecondaryAbi": ["armeabi-v7a","armeabi"],
    "HostArch": "x86_64",
    "HostSecondaryArch": "x86",
    "HostStaticBinaries": false,
    "CrossHost": "windows",
    "CrossHostArch": "x86",
    "CrossHostSecondaryArch": "x86_64",
    "ResourceOverlays": ["device/google/crosshatch/overlay","device/google/crosshatch/crosshatch/overlay"],
    "EnforceRROTargets": ["framework-res"],
    "EnforceRROExcludedOverlays": [],
    "AAPTCharacteristics": "nosdcard",
    "AAPTConfig": ["en_US,en_US,af_ZA,am_ET,ar_EG,ar_XB,as_IN,az_AZ,be_BY,bg_BG,bn_BD,bs_BA,ca_ES,cs_CZ,da_DK,de_DE,el_GR,en_AU,en_CA,en_GB,en_IN,en_XA,en_XC,es_ES,es_US,et_EE,eu_ES,fa_IR,fi_FI,fr_CA,fr_FR,gl_ES,gu_IN,hi_IN,hr_HR,hu_HU,hy_AM,in_ID,is_IS,it_IT,iw_IL,ja_JP,ka_GE,kk_KZ,km_KH,kn_IN,ko_KR,ky_KG,lo_LA,lt_LT,lv_LV,mk_MK,ml_IN,mn_MN,mr_IN,ms_MY,my_MM,nb_NO,ne_NP,nl_NL,or_IN,pa_IN,pl_PL,pt_BR,pt_PT,ro_RO,ru_RU,si_LK,sk_SK,sl_SI,sq_AL,sr_Latn_RS,sr_RS,sv_SE,sw_TZ,ta_IN,te_IN,th_TH,tl_PH,tr_TR,uk_UA,ur_PK,uz_UZ,vi_VN,zh_CN,zh_HK,zh_TW,zu_ZA,normal"],
    "AAPTPreferredConfig": "560dpi",
    "AAPTPrebuiltDPI": ["xxxhdpi","xxhdpi","xhdpi","hdpi"],
    "DefaultAppCertificate": "",
    "AppsDefaultVersionName": "9",
    "SanitizeHost": [],
    "SanitizeDevice": [],
    "SanitizeDeviceDiag": [],
    "SanitizeDeviceArch": [],
    "Safestack": false,
    "EnableCFI": true,
    "CFIExcludePaths": [],
    "CFIIncludePaths": ["device/google/cuttlefish_common/guest/libs/wpa_supplicant_8_lib","device/google/wahoo/wifi_offload","external/tinyxml2","external/wpa_supplicant_8","frameworks/av/camera","frameworks/av/media","frameworks/av/services","frameworks/minikin","hardware/broadcom/wlan/bcmdhd/wpa_supplicant_8_lib","hardware/interfaces/nfc","hardware/qcom/wlan/qcwcn/wpa_supplicant_8_lib","harware/interfaces/keymaster","system/bt","system/chre","system/core/libnetutils","system/core/libziparchive","system/gatekeeper","system/keymaster","system/nfc","system/security"],
    "IntegerOverflowExcludePaths": [],
    "ClangTidy": false,
    "TidyChecks": "",
    "NativeCoverage": false,
    "CoveragePaths": [],
    "CoverageExcludePaths": [],
    "ArtUseReadBarrier": true,
    "Binder32bit": false,
    "Brillo": false,
    "BtConfigIncludeDir": "device/google/crosshatch/bluetooth",
    "Device_uses_hwc2": true,
    "DeviceKernelHeaders": ["device/google/crosshatch/sdm845/kernel-headers"],
    "DevicePrefer32BitExecutables": false,
    "DeviceVndkVersion": "current",
    "Platform_vndk_version": "28",
    "ExtraVndkVersions": [],
    "DeviceSystemSdkVersions": ["28"],
    "Platform_systemsdk_versions": ["28"],
    "Malloc_not_svelte": true,
    "Override_rs_driver": "libRSDriver_adreno.so",
    "Treble_linker_namespaces": true,
    "Enforce_vintf_manifest": true,
    "Uml": false,
    "Use_lmkd_stats_log": true,
    "VendorPath": "vendor",
    "OdmPath": "odm",
    "ProductPath": "product",
    "MinimizeJavaDebugInfo": false,
    "UseGoma": false,
    "Arc": false,
    "DistDir": "",
    "NamespacesToExport": ["device/google/crosshatch/pixelstats","device/google/crosshatch/usb","device/google/crosshatch/health","hardware/google/av","hardware/google/interfaces","hardware/qcom/sdm845","vendor/qcom/sdm845"],
    "PgoAdditionalProfileDirs": [],
    "VendorVars": {
    }
}
Relavent enviroment variables in my shell

$ env
ANDROID_DEV_SCRIPTS=/home/mouath/dev/build/development/scripts:/home/mouath/dev/build/prebuilts/devtools/tools:/home/mouath/dev/build/external/selinux/prebuilts/bin:/home/mouath/dev/build/prebuilts/misc/linux-x86/dtc:/home/mouath/dev/build/prebuilts/misc/linux-x86/libufdt
LC_ALL=C
ANDROID_HOME=/opt/android-sdk
TARGET_PRODUCT=aosp_crosshatch
OLDPWD=/home/mouath/dev/build/prebuilts
OUT=/mnt/android/hello/build/target/product/crosshatch
JAVA_HOME=/home/mouath/dev/build/prebuilts/jdk/jdk9/linux-x86
BUILD_ENV_SEQUENCE_NUMBER=13
VIRTUAL_ENV=/home/mouath/dev/build/venv
ANDROID_TOOLCHAIN_2ND_ARCH=/home/mouath/dev/build/prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/bin
ANDROID_NDK=/opt/android-ndk
ANDROID_PRE_BUILD_PATHS=/home/mouath/dev/build/prebuilts/jdk/jdk9/linux-x86/bin:
TARGET_BUILD_VARIANT=userdebug
ANDROID_HOST_OUT_TESTCASES=/mnt/android/hello/build/host/linux-x86/testcases
GRADLE_HOME=/usr/share/java/gradle
ASAN_OPTIONS=detect_leaks=0
PWD=/home/mouath/dev/build
ANDROID_JAVA_TOOLCHAIN=/home/mouath/dev/build/prebuilts/jdk/jdk9/linux-x86/bin
CCACHE_DIR=/tmp/ccache
USE_CCACHE=1
ANDROID_TARGET_OUT_TESTCASES=/mnt/android/hello/build/target/product/crosshatch/testcases
ANDROID_BUILD_TOP=/home/mouath/dev/build
ANDROID_JACK_VM_ARGS=-Dfile.encoding=UTF-8 -XX:+TieredCompilation -Xmx4G
TARGET_BUILD_TYPE=release
OUT_DIR_COMMON_BASE=/mnt/android/hello
ANDROID_BUILD_PATHS=/mnt/android/hello/build/soong/host/linux-x86/bin:/mnt/android/hello/build/host/linux-x86/bin:/home/mouath/dev/build/prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/bin:/home/mouath/dev/build/prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/bin:/home/mouath/dev/build/development/scripts:/home/mouath/dev/build/prebuilts/devtools/tools:/home/mouath/dev/build/external/selinux/prebuilts/bin:/home/mouath/dev/build/prebuilts/misc/linux-x86/dtc:/home/mouath/dev/build/prebuilts/misc/linux-x86/libufdt:/home/mouath/dev/build/prebuilts/android-emulator/linux-x86_64:
XDG_SEAT_PATH=/org/freedesktop/DisplayManager/Seat0
CCACHE_PATH=/usr/bin
PYTHONPATH=/home/mouath/dev/build/development/python-packages:/home/mouath/dev/build/development/python-packages:/home/mouath/dev/build/development/python-packages:/home/mouath/dev/build/development/python-packages:
ANDROID_EMULATOR_PREBUILTS=/home/mouath/dev/build/prebuilts/android-emulator/linux-x86_64
ANDROID_PRODUCT_OUT=/mnt/android/hello/build/target/product/crosshatch
ANDROID_TOOLCHAIN=/home/mouath/dev/build/prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/bin
ANDROID_JAVA_HOME=/home/mouath/dev/build/prebuilts/jdk/jdk9/linux-x86
ANDROID_HOST_OUT=/mnt/android/hello/build/host/linux-x86

Emulator

Emulator had no screen output. I assumed it was expected.

logcat from emulator:

$ cat /mnt/android/hello/build/target/product/crosshatch/tmpAdbCmds/adbcommand7c7f3c71-d9d7-4f6e-ac3a-a66785f6ea98

adbcommand7c7f3c71-d9d7-4f6e-ac3a-a66785f6ea98.txt

Notes:

Behavior on the actual device

I flashed the image from within the build enviorment to my device. booted completely was able to unlock the screen and use it

fastboot

$ fastboot flashall -w
< waiting for any device >
--------------------------------------------
Bootloader Version...: b1c1-0.1-5034669
Baseband Version.....: g845-00023-180917-B-5014671
Serial Number........: 8BGY0Z66E
--------------------------------------------
Checking product
OKAY [  0.060s]
Checking version-bootloader
OKAY [  0.060s]
Checking version-baseband
OKAY [  0.060s]
target reported max download size of 268435456 bytes
Erase successful, but not automatically formatting.
File system type raw not supported.
Erase successful, but not automatically formatting.
File system type raw not supported.
Sending 'boot_a' (65536 KB)...
OKAY [  1.680s]
Writing 'boot_a'...
OKAY [  0.323s]
Sending 'dtbo_a' (8192 KB)...
OKAY [  0.297s]
Writing 'dtbo_a'...
OKAY [  0.096s]
Sending 'product_a' (4956 KB)...
OKAY [  0.214s]
Writing 'product_a'...
OKAY [  0.160s]
Sending sparse 'system_a' 1/5 (262140 KB)...
OKAY [  6.570s]
Writing 'system_a' 1/5...
OKAY [  0.063s]
Sending sparse 'system_a' 2/5 (262140 KB)...
OKAY [  6.527s]
Writing 'system_a' 2/5...
OKAY [  0.063s]
Sending sparse 'system_a' 3/5 (262140 KB)...
OKAY [  6.447s]
Writing 'system_a' 3/5...
OKAY [  0.063s]
Sending sparse 'system_a' 4/5 (262140 KB)...
OKAY [  6.547s]
Writing 'system_a' 4/5...
OKAY [  0.063s]
Sending sparse 'system_a' 5/5 (188280 KB)...
OKAY [  4.717s]
Writing 'system_a' 5/5...
OKAY [  0.900s]
Sending 'system_b' (89388 KB)...
OKAY [  2.290s]
Writing 'system_b'...
OKAY [  0.550s]
Sending 'vbmeta_a' (4 KB)...
OKAY [  0.120s]
Writing 'vbmeta_a'...
OKAY [  0.067s]
Sending sparse 'vendor_a' 1/2 (262140 KB)...
OKAY [  6.533s]
Writing 'vendor_a' 1/2...
OKAY [  0.366s]
Sending sparse 'vendor_a' 2/2 (220856 KB)...
OKAY [  5.494s]
Writing 'vendor_a' 2/2...
OKAY [  1.310s]
Setting current slot to 'a'...
OKAY [  0.080s]
Erasing 'userdata'...
OKAY [  9.787s]
Erasing 'metadata'...
OKAY [  0.133s]
Rebooting...

Finished. Total time: 63.972s

here's a list of what worked and what not (I was not thorough):

Works:

Does NOT work:

Logs

logcat

adb logcat -d logcat.txt

logcat when trying to use phone app and crashing:

adb logcat -d logcat-crash-phone.txt

logcat when trying to activate the WiFi:

adb logcat -d logcat-crash-wifi.txt

getprop

adb shell getprop getprop.txt

dumpstate

adb shell dumpstate dumpstate.txt.zip

dumpysys

adb shell dumpsys dumpsys.txt

dumpsys gfxinfo

adb shell dumpsys gfxinfo gfxinfo-gpu.txt

packages

adb shell pm list packages packages.txt

ls

adb shell ls -al ls.txt

netstat

adb shell netstat netstat.txt

ps

adb shell ps ps.txt

top

adb shell top top.txt

Environment

Final note:

I wanted to provide you with as much information, all the build folders are still there I just don't know how to navigate them/extract meaningful information.

I'm willing to redo the build and test it again granted that you provide me with some criterea to what to test and what logs you are intrested in.

Cheers and thank you.

dan-v commented 5 years ago

wow - @iMouath that's a very detailed report! I have a Pixel 3 XL using android-prepare-vendor (quite a few commits behind https://github.com/RattlesnakeOS/android-prepare-vendor but no changes) but it results in working audio and wifi. I haven't tried pulling in latest commits from android-prepare-vendor and testing though. Might be worth testing back on the same commit that I'm on.

iMouath commented 5 years ago

It's very likely that I botched the build.

I'll try and read more about the build process, I'm confused about the https://developers.google.com/android/drivers I didn't use them which could explain why I'm missing hardware functionality.

Update: I've rebuild the image using android-9.0.0_r30 branch and made sure the tool output to the aosp source root. flashed the image but encountered the same behavior.

anestisb commented 5 years ago

@iMouath really appreciate the very detailed report. Will start working on them.

For WiFi & Audio I've created separate tasks to track them (see https://github.com/anestisb/android-prepare-vendor/issues/150 & https://github.com/anestisb/android-prepare-vendor/issues/151).

eSIM should be available if you use the full config: https://github.com/anestisb/android-prepare-vendor/blob/7dee0eaf3ad65bb34be68fe6fe2922af122990ea/crosshatch/config.json#L230

For the assistance thing I'm not sure where it's implemented and if its something can be made available without the Google App blobs. Will investigate, with low priority though.

iMouath commented 5 years ago

@anestisb I'm glad the report was useful to you.

I'll build another image using the full config when I get a chance and give you feedback on it.

regarding the assistance squeeze thingy (Active Edge) I suspect it's packaged under SystemUIGoogle.apk but as you said it isn't a core feature and a blob so not really a concern at all. and it's reported to be broken in all AOSP roms.