Consider using lineage extract scripts

[chirayudesai]

We maintain a set of scripts here, that are eventually used to produce the same end-result.

We could consolidate some of these efforts by using the lineage bits where possible.

Example fillelists:

• https://github.com/LineageOS/android_device_google_crosshatch/blob/lineage-17.1/crosshatch/device-proprietary-files.txt
• https://github.com/LineageOS/android_device_google_crosshatch/blob/lineage-17.1/crosshatch/device-proprietary-files-other.txt

Script:

• https://github.com/LineageOS/android_vendor_lineage/blob/lineage-18.0/build/tools/extract_utils.sh

It's bash scripts replaced with bash scripts, but it's a set of bash scripts we use and maintain vs a set of bash scripts a lot more people use and maintain

What we could do is have apv fetch the factory images, extract them, and present the directories to the script linked above.

At this point, somehow it feels like it'd be easier to list what to include than what to exclude. Besides, we could also base our list off their lists and work from that anyway. Those lists are equivalent to the full variant here, but we can trim it down to naked for the base, and maybe for full we just use that wholesale.

Thoughts?

[dan-v]

I'm onboard with dropping apv in favor of this approach.

[thestinger]

Originally, they weren't assembling their own vendor but rather they were using the stock vendor image.

Also, it's incorrect to simply bundle an apk that has been through dex processing specific to the build for the device. android-prepare-vendor largely came into existence to work around the issue of dex being stripped out of the apks and only preoptimized code being available. That's generally not as much of an issue anymore but that's why android-prepare-vendor has the infrastructure for decompiling / extracting dex code and reverting various forms of optimized dex code back to the proper format for an apk to be installed or included in an OS build. I think it's still an issue to some extent and android-prepare-vendor isn't just a no-op for those.

[thestinger]

So, the hardest problem solved by apv may not actually be an issue anymore. Back when it was a new project, it was the only realistic way to do fully signed, production builds with robust handling of Java device support code in system without sources in AOSP (doesn't necessarily mean no sources available). Treble has made things a lot cleaner, and the move away from full AOT compilation to JIT + JIT-assisted AOT compilation led to them moving away from including preoptimized code for everything in the OS + stripping out dex code. It is still done to some extent, mostly for libraries, but it may be increasingly less of an issue due to Treble since vendor doesn't go through preoptimization - only system, and most things that are device specific have been correctly moved to vendor.

[chirayudesai]

Current what I intend to do is:

• Create a vendor/pixel
• Add a simple wrapper script that takes device name and build id, and passes that to the existing apv download scripts
• Extract downloaded images using existing apv scripts
• Pass extracted image to lineage scripts to create vendor
• Add firmware, and other bits we have (build_id, board-info) - all via existing apv scripts

For the file list, it may make sense to start with the android10 lists, should be easy to generate based on the lineage lists + what we have currently in apv, since Lineage lists basically just list all the files included, with special handling for some (PRESIGNED apks, shared libs used by oss components, etc)

Pros:

• Sharing scripts thus not having to maintain them entirely on our own
• Sharing lists potentially, Lineage lists are formatted better, and I think ours end up being a subset of theirs anyway for a naked config.

Cons:

• Gotta write / wrap some scripts and test, takes time.

We'd have to make plenty of updates for 11.0 anyway from the looks of it, this way we can use that effort on something common and hopefully avoid having to do this for the next release / future updates / devices.

[chirayudesai]

Originally, they weren't assembling their own vendor but rather they were using the stock vendor image.

For the pixels yeah before they were officially shipping, but other devices use the same scripts and that's been done since forever.

Also, it's incorrect to simply bundle an apk that has been through dex processing specific to the build for the device. android-prepare-vendor largely came into existence to work around the issue of dex being stripped out of the apks and only preoptimized code being available. That's generally not as much of an issue anymore but that's why android-prepare-vendor has the infrastructure for decompiling / extracting dex code and reverting various forms of optimized dex code back to the proper format for an apk to be installed or included in an OS build. I think it's still an issue to some extent and android-prepare-vendor isn't just a no-op for those.

Lineage scripts do handle oat2dex anyway, and if anything is missing we could just bring over the apv support, and I feel like that would still be saner than trying to update apv for 11.0 (vendor-skip-files is a big mess and it's only getting messier and I'd really like to avoid that whole thing)

[thestinger]

Yeah, that makes sense. I'm worried about this stuff getting finished relatively promptly and being in a state that's good enough to be usable.

[chirayudesai]

I created https://github.com/AOSPAlliance/vendor_pixel and https://github.com/AOSPAlliance/prebuilts_tools-pixel

Imported extract_utils.sh and other tools around it with full history and same paths so we can pick commits easily in the future.

Next steps:

• Downloading of AOSP images (one script)
• Extraction of AOSP images (debugfs is what we use, should add to extract_utils.sh and send upstream)
• Setup of radio images (one script)

And then a common wrapper script that calls them in order.