[10.0][4.9][Tone/Dora] Build error

[SoftReaper]

Platform: Tone Device: Dora Kernel version: 4.9 Android version: 10.0 branch r26 Software binaries version: none at this time, but planning SW_BINARIES_FOR_XPERIA_ANDROID_9.0_2.3.2_V9_TONE

Description Using repo r26 while reading the official documentation but using "android-10_legacy" branch for "local_manifests" to use 4.9 Kernel and avoid errors from trying to get a brebuilt 4.14 kernel that doesn't exists for my testing device (lack of info about legacy status of MSM8996 and 4.14 downloadable binaries is missleading, i've found informations only after reading local manifest between branch r15 and r21 comments)

selected "10" F8131-userdebug for target build

While trying to build with limited hardware support, i get error about missing kernel (it seems that script not copying " " to out/target/sony/dora/kernel)

FAILED: ninja: 'out/target/product/dora/kernel', needed by 'out/target/product/dora/boot.img', missing and no known rule to make it FAILED: [W][2020-02-29T12:18:30+0100][122637] void cmdline::logParams(nsjconf_t *)():250 Process will be UID/EUID=0 in the global user namespace, and will have user root-level access to files

as the system didn't compiled anything, i tried:

cp kernel/sony/msm-4.9/common-kernel/kernel-dtb-dora /out/target/product/dora/kernel

and restarted with make -j&(nproc)

by doing this, building process actually started, before crashing:

see log at the end of the post

Symptoms So,

• prebuilt kernel not copying to output dir
• missing qualcomm related dependencies but, there is regularly troubles with Qcamera2, so any advice would be appreciated

How to reproduce Sync Tree with branch "android-10.0.0_r26" and Local_manifests branch "android-10_legacy", and follow the official guide, use "" device identifier with "source build/envsetup.sh && lunch"

Build error:

FAILED: out/target/product/dora/obj_arm/SHARED_LIBRARIES/libmmcamera_interface_intermediates/src/mm_camera_stream.o
/bin/bash -c "PWD=/proc/self/cwd  prebuilts/clang/host/linux-x86/clang-r353983c/bin/clang       -I system/media/camera/include -I vendor/qcom/opensource/camera/QCamera2/stack/mm-camera-interface/inc -I vendor/qcom/opensource/camera/QCamera2/stack/mm-camera-interface/../common -I vendor/qcom/opensource/camera/QCamera2/stack/mm-camera-interface/../common/leak -I vendor/qcom/opensource/camera/QCamera2/stack/mm-camera-interface -I out/target/product/dora/obj_arm/SHARED_LIBRARIES/libmmcamera_interface_intermediates -I out/target/product/dora/gen/SHARED_LIBRARIES/libmmcamera_interface_intermediates \$(cat out/target/product/dora/obj_arm/SHARED_LIBRARIES/libmmcamera_interface_intermediates/import_includes)   -isystem out/target/product/dora/obj/include -c  -fomit-frame-pointer -DANDROID -fmessage-length=0 -W -Wall -Wno-unused -Winit-self -Wpointer-arith -no-canonical-prefixes -DNDEBUG -UDEBUG -fno-exceptions -Wno-multichar -O2 -g -fno-strict-aliasing -fdebug-prefix-map=/proc/self/cwd= -D__compiler_offsetof=__builtin_offsetof -faddrsig -Wimplicit-fallthrough -Werror=int-conversion -Wno-reserved-id-macro -Wno-format-pedantic -Wno-unused-command-line-argument -fcolor-diagnostics -Wno-zero-as-null-pointer-constant -Wno-sign-compare -Wno-defaulted-function-deleted -Wno-inconsistent-missing-override -ffunction-sections -fdata-sections -fno-short-enums -funwind-tables -fstack-protector-strong -Wa,--noexecstack -D_FORTIFY_SOURCE=2 -Wstrict-aliasing=2 -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Werror=date-time -Werror=format-security -nostdlibinc -msoft-float -march=armv8-a -mfloat-abi=softfp -mfpu=neon-fp-armv8 -mcpu=cortex-a53 -mfpu=neon-fp-armv8 -D__ARM_FEATURE_LPAE=1 -target armv7a-linux-androideabi -Bprebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/arm-linux-androideabi/bin   -std=gnu99 -mthumb -Os   -DSYSTEM_HEADER_PREFIX=sys -DUSE_ION -DVENUS_PRESENT -DUBWC_PRESENT -D_ANDROID_ -DQCAMERA_REDEFINE_LOG -DFDLEAK_FLAG -DMEMLEAK_FLAG -DCAMERA_ION_HEAP_ID=ION_IOMMU_HEAP_ID -Wall -Wextra -Werror -DUSE_4_9_DEFS -D__ANDROID_API__=29 -D__ANDROID_VNDK__ -fPIC -D_USING_LIBCXX -DANDROID_STRICT   -Werror=int-to-pointer-cast -Werror=pointer-to-int-cast -Werror=address-of-temporary -Werror=return-type -Wno-tautological-constant-compare -Wno-tautological-type-limit-compare -Wno-tautological-unsigned-enum-zero-compare -Wno-tautological-unsigned-zero-compare -Wno-c++98-compat-extra-semi -Wno-return-std-move-in-c++11 -MD -MF out/target/product/dora/obj_arm/SHARED_LIBRARIES/libmmcamera_interface_intermediates/src/mm_camera_stream.d -o out/target/product/dora/obj_arm/SHARED_LIBRARIES/libmmcamera_interface_intermediates/src/mm_camera_stream.o vendor/qcom/opensource/camera/QCamera2/stack/mm-camera-interface/src/mm_camera_stream.c"
vendor/qcom/opensource/camera/QCamera2/stack/mm-camera-interface/src/mm_camera_stream.c:36:10: fatal error: 'media/msm_media_info.h' file not found
#include <media/msm_media_info.h>
         ^~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
[ 20% 24757/119535] //bionic/libc:libc.llndk versioner preprocess include
warning: attempted to generate guard with empty availability: obsoleted = 21
warning: attempted to generate guard with empty availability: obsoleted = 23
FAILED: [W][2020-02-29T12:30:53+0100][191963] void cmdline::logParams(nsjconf_t *)():250 Process will be UID/EUID=0 in the global user namespace, and will have user root-level access to files
[W][2020-02-29T12:30:53+0100][191963] void cmdline::logParams(nsjconf_t *)():260 Process will be GID/EGID=0 in the global user namespace, and will have group root-level access to files
12:36:52 ninja failed with: exit status 1

#### failed to build some targets (10:28 (mm:ss)) ####```

[MarijnS95]

While trying to build with limited hardware support, i get error about missing kernel (it seems that script not copying " " to out/target/sony/dora/kernel)

This is likely because the kernel version is still at 4.14, see https://github.com/sonyxperiadev/device-sony-tone/pull/190/files

Those headers are only available for 4.9, while the headers for 4.14 are selected during compilation due to that same variable. Try picking that linked pull request and build again.

[SoftReaper]

surely, it'll help ^^

edit: gonna try now, thank you by advance, i'll update as soon as things are done

[SoftReaper]

Sorry for double posting, your suggestion solve effectively the "missing kernel" problem, but not missing files:

FAILED: out/target/product/dora/obj_arm/SHARED_LIBRARIES/camera.msm8996_intermediates/HAL/QCameraPostProc.o
/bin/bash -c "PWD=/proc/self/cwd  prebuilts/clang/host/linux-x86/clang-r353983c/bin/clang++     -I vendor/qcom/opensource/camera/QCamera2/../mm-image-codec/qexif -I vendor/qcom/opensource/camera/QCamera2/../mm-image-codec/qomx_core -I vendor/qcom/opensource/camera/QCamera2/include -I vendor/qcom/opensource/camera/QCamera2/stack/common -I vendor/qcom/opensource/camera/QCamera2/stack/common/leak -I vendor/qcom/opensource/camera/QCamera2/stack/mm-camera-interface/inc -I vendor/qcom/opensource/camera/QCamera2/util -I vendor/qcom/opensource/camera/QCamera2/HAL3 -I hardware/qcom/media/sdm660-libion/libstagefrighthw -I hardware/qcom/media/sdm660-libion/mm-core/inc -I system/core/libion/kernel-headers -I system/core/libion/include -I vendor/qcom/opensource/camera/QCamera2/HAL -I out/target/product/dora/obj/include/qcom/display -I hardware/qcom/display/sde/libqservice -I vendor/qcom/opensource/camera/QCamera2 -I out/target/product/dora/obj_arm/SHARED_LIBRARIES/camera.msm8996_intermediates -I out/target/product/dora/gen/SHARED_LIBRARIES/camera.msm8996_intermediates \$(cat out/target/product/dora/obj_arm/SHARED_LIBRARIES/camera.msm8996_intermediates/import_includes)   -isystem out/target/product/dora/obj/include -isystem kernel/sony/msm-4.9/common-headers/kernel-headers -c  -fomit-frame-pointer -DANDROID -fmessage-length=0 -W -Wall -Wno-unused -Winit-self -Wpointer-arith -no-canonical-prefixes -DNDEBUG -UDEBUG -fno-exceptions -Wno-multichar -O2 -g -fno-strict-aliasing -fdebug-prefix-map=/proc/self/cwd= -D__compiler_offsetof=__builtin_offsetof -faddrsig -Wimplicit-fallthrough -Werror=int-conversion -Wno-reserved-id-macro -Wno-format-pedantic -Wno-unused-command-line-argument -fcolor-diagnostics -Wno-zero-as-null-pointer-constant -Wno-sign-compare -Wno-defaulted-function-deleted -Wno-inconsistent-missing-override -ffunction-sections -fdata-sections -fno-short-enums -funwind-tables -fstack-protector-strong -Wa,--noexecstack -D_FORTIFY_SOURCE=2 -Wstrict-aliasing=2 -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Werror=date-time -Werror=format-security -nostdlibinc -msoft-float -march=armv8-a -mfloat-abi=softfp -mfpu=neon-fp-armv8 -mcpu=cortex-a53 -mfpu=neon-fp-armv8 -D__ARM_FEATURE_LPAE=1 -target armv7a-linux-androideabi -Bprebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/arm-linux-androideabi/bin  -Wsign-promo -D_LIBCPP_ENABLE_THREAD_SAFETY_ANNOTATIONS -Wno-thread-safety-negative -Wno-gnu-include-next -fvisibility-inlines-hidden  -std=gnu++17 -mthumb -Os  -fno-rtti -Wall -Wextra -Werror -DFDLEAK_FLAG -DMEMLEAK_FLAG -DQCAMERA_HAL1_SUPPORT -DSYSTEM_HEADER_PREFIX=sys -DHAS_MULTIMEDIA_HINTS -D_ANDROID -DVANILLA_HAL -DUSE_DISPLAY_SERVICE -std=c++11 -std=gnu++1y -DDEFAULT_DENOISE_MODE_ON -DHAL3 -DQCAMERA_REDEFINE_LOG -DVENUS_PRESENT -DUBWC_PRESENT -DTARGET_MSM8996 -DUSE_CAMERA_METABUFFER_UTILS -D__ANDROID_API__=29 -D__ANDROID_VNDK__ -fPIC -D_USING_LIBCXX -DANDROID_STRICT   -Werror=int-to-pointer-cast -Werror=pointer-to-int-cast -Werror=address-of-temporary -Werror=return-type -Wno-tautological-constant-compare -Wno-tautological-type-limit-compare -Wno-tautological-unsigned-enum-zero-compare -Wno-tautological-unsigned-zero-compare -Wno-c++98-compat-extra-semi -Wno-return-std-move-in-c++11  -MD -MF out/target/product/dora/obj_arm/SHARED_LIBRARIES/camera.msm8996_intermediates/HAL/QCameraPostProc.d -o out/target/product/dora/obj_arm/SHARED_LIBRARIES/camera.msm8996_intermediates/HAL/QCameraPostProc.o vendor/qcom/opensource/camera/QCamera2/HAL/QCameraPostProc.cpp"
In file included from vendor/qcom/opensource/camera/QCamera2/HAL/QCameraPostProc.cpp:39:
In file included from vendor/qcom/opensource/camera/QCamera2/HAL/QCamera2HWI.h:40:
In file included from vendor/qcom/opensource/camera/QCamera2/HAL/QCameraChannel.h:34:
vendor/qcom/opensource/camera/QCamera2/HAL/QCameraMem.h:43:10: fatal error: 'OMX_QCOMExtns.h' file not found
#include "OMX_QCOMExtns.h"
         ^~~~~~~~~~~~~~~~~
1 error generated.
FAILED: [W][2020-02-29T15:08:39+0100][76059] void cmdline::logParams(nsjconf_t *)():250 Process will be UID/EUID=0 in the global user namespace, and will have user root-level access to files
[W][2020-02-29T15:08:39+0100][76059] void cmdline::logParams(nsjconf_t *)():260 Process will be GID/EGID=0 in the global user namespace, and will have group root-level access to files
15:29:52 ninja failed with: exit status 1

#### failed to build some targets (26:58 (mm:ss)) ####```

[MarijnS95]

Sorry for double posting

Put the log in triple backticks, not single ones. Also, you can use a details section in markdown to make it collabsible. And since the error is the same for every file, there's no need to post the entire thing.

What media HAL are you building with? That one is responsible for exporting that header.

[SoftReaper]

For the time being, i don't even pretend to choose anything, i've just started 2 or 3 days ago to thinker and start learning about AOSP, i've started by simple compiling of SODP before starting to modify anything to see how things are working

so, "What media HAL are you building with?", i seriously don't have any idea as i'm simply trying to follow the official SODP android 10 build guide. Didn't thought this will lead to all those things as Android Pie goes without any troubles (running on my phone at this moment)

i've reduced the amount of garbage in my previous post and i'll do the same one the OP for better readability

[MarijnS95]

Didn't thought this will lead to all those things as Android Pie goes without any troubles

We hoped so too, but as it turns out we keep supporting old devices while new ones show up like it's Christmas. At this point we're with a far too small team of open source contributors (people that spend their free time) to properly maintain everything. That said, Loire and Tone are old enough to not receive updated closed source binaries anymore, preventing vital components such as keymaster from working under the new kernel. From that point on supporting multiple platforms on different kernels, including different HAL versions, becomes plain madness.

device/sony/common/hardware/qcom/Android.mk selects which media HAL is used. Can you check what it sets? It should be sdm660-libion afaik (which doesn't compile under 4.9, see https://github.com/sonyxperiadev/device-sony-loire/issues/263) though at least export the necessary header.

[SoftReaper]

ifneq ($(filter $(QCOM_NEW_MEDIA_PLATFORM), $(TARGET_BOARD_PLATFORM)),) QCOM_MEDIA_ROOT := hardware/qcom/media/sm8150 else QCOM_MEDIA_ROOT := **hardware/qcom/media/sdm660-libion** endif

it looks you're right

i really understand your point, but, like many people "starting" their dive in the open source world, i've though that "big companies" have a, even minimalistic, team on that kind of projects, even if it's just for PR, but i know that most, if not almost every peoples enrolled in theses programs and communities, are doing this on their spare time, and i'm gracefull to them, as most innovations comes from them and their devotion

UPDATE: After Applying ix5 update to Tone target: https://github.com/sonyxperiadev/device-sony-tone/pull/190/files

and reverted this one: https://github.com/sonyxperiadev/device-sony-common/commit/64cfaf3715ec64e040dc978de7fcba4f07807b04

Everything went fine until last 2% where i got an Out Of Memory error, the system is full of an impressive number of sleeping thread eating the 8GB Main memory and 12GB of swap, i'll try to assign 12GB of RAM to the VM and 50GB of swap on NVME Raid0 and retry

[MarijnS95]

@SoftReaper It is quite a coincidence that I opened this issue after your edit. If you have new info, hours after the original post, be sure to put it into a new comment so that notifications are distributed. Otherwise your update might unintentionally go completely ignored.

Android has a fun component that does API checks, plays with documentation and whatnot which takes insane amounts of RAM. All I can recommend is to run with less threads (-j1 perhaps) and retry. You might want to check out he Optimization Guide from @ix5 to disable these API checks, but keep in mind the orange warning about applying a patch on Android 10.

[SoftReaper]

At this moment i'm at j18, 11GB of Ram and 50GB of swap, i'll try it single threaded and read from the begining the ix5's guide, as it should be an interesting and usefull read, i'll keep you updated.

Don't worry about "edits and updates" when there is something really important, i'm posting in a new post, i'm editing only if it's just informative or possibly usefull for future readers

[jaylinski]

@SoftReaper I have the same OOM issue on a Loire build (https://github.com/sonyxperiadev/device-sony-loire/issues/263). I thought this issue was caused by the JVM memroy limit (heap size is limited to 2G on my machine). I'll also try to build with only 1 CPU.

[MarijnS95]

@jaylinski @SoftReaper There are various OOM issues, apparently. I have never seen this one where a JVM allocation fails, which looks to be that explicit heap limit.

For me it usually freezes up the entire machine while writing to swap (16GB mem, 8GB swap) before eventually continuing or crashing in a random apk compilation. The difference might be that I have disabled that metalava apicheck/doc thingy.

[SoftReaper]

So, get back home and saw:

#### build completed successfully (03:10:22 (hh:mm:ss)) ####

So, builded with -j18 (AMD TR1920x) 11GB of memory/50GB of Swap and 18 Threads on Hyper-V VM

No idea of how much have been used but while monitoring my previous try, i used way more than 20GB (swap+ram)

didn't disabled anything or changed JVM configuration, so just lack of memory/swap i think, i'm going to try that build on the device

[SoftReaper]

after a bit, can't pass sony logo, no boot animation and pretty hot device, so it's a miss

before first boot with twrp, data is readable but after first failed boot not anymore, so the system could be working, but without graphics (that could be linked to the reverted changes and mods applied on my upper post to start compilation)

i'll make a try tomorow by following the complete ix5 guide, i'll keep you posted, anyway i want to thank all of you for the time taken to give me help, advice and support ;)

[Haxk20]

@SoftReaper As you say before first boot data is readable. Did you flash the userdata.img ? Its mandatory to flash it when going from android 9 to 10. Failure to do so will resoult in corrupted data and you will have to flash the userdata.img to fix it.

[MarijnS95]

@SoftReaper Is adb available? No bootimage but a stuck device indeed sounds like broken graphics/display HAL. Can you validate hardware/qcom/display/sde is checked out at LA.UM.7.3...?

@Haxk20 Apart changes in Android itself there should be no reason to wipe userdata. The wipe you are talking about is due to changes in Keymaster when going from kernel 4.9 to 4.14. Loire and Tone use a different keymaster anyway (which is the main reason 4.14 isn't as easy as it is for the other devices).

[Haxk20]

Oh i just read the tag for the issue. This is purely 4.9 then yeah no need to wipe data. Even tho it should still be good to do so. @MarijnS95 Also not passing SONY logo sounds like a bootimage issue to me mostly as broken display HAL would at least allow the boot to blank out the display at start of boot process. It seems like the device is not even getting there by what i see.

after a bit, can't pass sony logo, no boot animation

[SoftReaper]

@MarijnS95 : i've thrown back the original firmware (my device is not a "thinkering" phone but my everyday one) i'll put 10 ont it later this day to verify your ADB theory, but considering the phone thermal shutdown on sony logo, i've got little hope that it'll only be a graphic driver issue

@Haxk20 : i always flash userdata to blank, too many bad remembers about my LG EVE and 2.3 cyanogen ...

As there is 4.9 Kernel on 9 and 10 i thought it should work with the previous one, so i've tried to flash the 9 kernel and Binaries and it end in bootloop with a reboot every 5s. I've also compared both boot images and the Pie is bigger than Q, as i thought there'll be more things in the later due to the higher feature count that could involve the kernel

[SoftReaper]

so, got ADB on 10 but not display, after a little git checkout remotes/sony/aosp/LA.UM.7.3.r1 and ./repo_update.sh, many modifications appened and some commit "discarded" (no details from the script)

i'll get back to you once build is finished

[SoftReaper]

Filesizes are stricly identical from first (no display) and second (downloading) builds, and got the same problem

I'll try, as advised previously, the ix5 repository and if still not working i'll give up

[chris42]

@SoftReaper not sure, if it helps, but I was trying 10/4.9 on kugo and had bootloops as well, because of a version bump in display composer. Had to revert that (commit 0634f99808583bac9586649da024c1e24d14964b) You might check the other reverts and PRs I did in #536 as well, maybe that is helping you.

[MarijnS95]

@SoftReaper With adb you can extract a logcat and dmesg. However, as @chris42 correctly mentioned we switched to a different composer that does not work with the 7.3 branches. Reverting that might get it going, but be sure to run make installclean to get rid of stray files.

@chris42 did a nice cleanup yesterday too, consolidating all outstanding "4.9 on Android 10" related issues into #536. Since you are both facing and fixing the same issues, would you mind closing this one @SoftReaper and continuing there?

Finally, the changes from @ix5 are being merged to legacy branches as we speak, so you should not need to revert anything anymore.

Follow the new branches here, and stay alert for other merges/pushes: https://github.com/sonyxperiadev/local_manifests/commits/android-10_legacy Most recently updated repos: https://github.com/sonyxperiadev?tab=repositories

[SoftReaper]

thank you for the update, i'll try all of that and keep you posted there

don't forget to add </manifest> at the end of the device.xml in local_manifests to avoid error while syncing until correction

[SoftReaper]

i've posted updates about my successfull build there https://github.com/sonyxperiadev/bug_tracker/issues/536#issuecomment-593805580

well, it finished, but with ceveats

[ix5]

This should be closed then.