Define a co-maintainership with Qualcomm, and migrate the layer to a Qualcomm hosted Github org

[ndechesne]

hey,

as discussed with a few people privately, we intend to migrate meta-qcom into a new Github org (most likely github.com/quic-yocto) in the near future. Qualcomm has now started to develop a BSP based on Yocto development branch, using linux-yocto kernel (e.g. meta-qcom-hwe layer), and there is no strong reason to maintain 2 competing Yocto BSP. We want to merge meta-qcom and meta-qcom-hwe 'master' branches and have all development done jointly on meta-qcom.

We have a rough goal for this migration to be done between Dec 24 and Feb 25.

There are many technical details to iron out, so let's use this issue to coordinate.

Here is a summary of what we've discussed so far

1. The migration should be implemented as a Github ownership transfer, with redirection, to minimize the chance of breaking users
2. meta-qcom is a BSP layer, and any policy/distro configs should be done outside of meta-qcom. Qualcomm maintains meta-qcom-distro for that purpose
3. We want to migrate the current master branch (and future yocto stable branch will follow from that), but we are not doing any retro-active changes. meta-qcom-hwe will continue to support QC Linux 1.x releases with the kirkstone branch, and all other branches on meta-qcom won't change.
4. CI: we will move away from current Linaro tuxmake based CI for master branch and use existing Meta-qcom-hwe (main) CI infrastructure.
5. CI: Keep Linaro CI for old branches (through tuxmake)
6. Machines. Meta-qcom has ‘generic’ machines only, and no plans to add board specific machines. Linaro will continue to maintain/support the generic machines in the merged layer. There are discussions/efforts to support genericarm64 and genericarm machines from meta-yocto-bsp, but for that this is in addition to meta-qcom generic machines. Qualcomm is interested in the generic machines from meta-yocto-bsp, in addition to the specific machine conf file currently in meta-qcom-hwe (which will migrate to meta-qcom)
7. Meta-qcom: find . -name *.bb | wc -l  108, let’s review them and remove unnecessary recipes (If any)
8. Understand what initramfs-firmware-image.bb and firmware-woa are, and how we want to support them after the merge.
9. Meta-qcom has support for meta-linux-mainline kernel recipes. is anyone using that? do we want to maintain them?
10. Meta-qcom maintains Mesa_git recipe. Is that still needed? For which purpose?
11. What are the Binary blobs which are needed in meta-qcom, and how do we handle their redistribution.
12. meta-qcom will be open for contributions, Qualcomm and Linaro will co-maintain and thus review external contributions.
13. meta-qcom-hwe supports 2 flavor of BSP (Base and Custom), and these 2 profiles will continue to exist.
14. Future Qualcomm Linux releases will be based off meta-qcom only (e.g. 2.x, 3.x..), and will be maintain by Qualcomm for productization/commercialization.

We will shortly open a branch (somewhere still TBD) on github.com/quic-yocto, to start working on this

[lumag]

Here is a list of all the recipes and appends in the meta-qcom layer together with their descriptions and some notes.

./recipes-bsp/hexagon-dsp-binaries/hexagon-dsp-binaries_20241017.bb

fastrpc_shell and libraries for the DSP, redistributable, covered by
LICENSE.qcom.txt

./recipes-bsp/firmware/firmware-ath6kl_git.bb

additional ath6k firmware, not suitable for linux-firmware, redistributable

./recipes-bsp/firmware/firmware-qcom-dragonboard410c-bootloader-sdcard_17.09.bb ./recipes-bsp/firmware/firmware-qcom-dragonboard845c_20190529180356-v4.bb ./recipes-bsp/firmware/firmware-qcom-rb3gen2_00039.2.bb

recipes to package redistributable bootloader and firmware binaries for
supported devices.

Note: DB845c archive and recipe also install firmware for the Renesas xHCI
PCI controller. It has been provided to us by Thundercomm in the same ZIP
archive and claimed to be covered by LICENSE.qcom.txt, but further
investigation could not prove this claim. It is being used by the layer on
the premises of being abandonware by the Renesas side, but we don't have a
good enough licence to get it into the linux-firmware. Maybe Qualcomm can
help with that by providing a bigger push on Renesas, especially granted
that RB3gen2 also uses the same chip (which actually requires the
firmware).

./recipes-bsp/firmware/firmware-qcom-dragonboard-apq8074.bb ./recipes-bsp/firmware/firmware-qcom-ifc6410.bb ./recipes-bsp/firmware/firmware-qcom-ifc6560.bb ./recipes-bsp/firmware/firmware-qcom-sm8150-hdk.bb ./recipes-bsp/firmware/firmware-qcom-sm8350-hdk.bb ./recipes-bsp/firmware/firmware-qcom-sm8450-hdk.bb ./recipes-bsp/firmware/firmware-qcom-sm8550-hdk.bb ./recipes-bsp/firmware/firmware-qcom-sm8650-hdk.bb

recipes to package non-redistributable binary firmware for supported
devices (HDKs, devboards). Requires the user to explicitly specify the
NON-HLOS.bin / adreno.tar.gz / dspso.bin (pending) in order to generate
non-empty packages.

./recipes-bsp/firmware-nexus/firmware-qcom-nexus4.bb ./recipes-bsp/firmware-nexus/firmware-qcom-nexus5.bb ./recipes-bsp/firmware-nexus/firmware-qcom-nexus5x.bb ./recipes-bsp/firmware-nexus/firmware-qcom-nexus6.bb ./recipes-bsp/firmware-nexus/firmware-qcom-nexus6p.bb ./recipes-bsp/firmware-nexus/firmware-qcom-nexus7-2013.bb ./recipes-bsp/firmware-nexus/firmware-qcom-pixel.bb ./recipes-bsp/firmware-nexus/firmware-qcom-pixel2.bb ./recipes-bsp/firmware-nexus/firmware-qcom-pixel3.bb ./recipes-bsp/firmware-nexus/firmware-qcom-pixel3a.bb ./recipes-bsp/firmware-nexus/firmware-qcom-pixel4.bb ./recipes-bsp/firmware-nexus/firmware-qcom-pixel4a-5g.bb ./recipes-bsp/firmware-nexus/firmware-qcom-pixel4a.bb ./recipes-bsp/firmware-nexus/firmware-qcom-pixel5.bb ./recipes-bsp/firmware-nexus/firmware-qcom-pixel5a-5g.bb

 recipes to package non-redistributable binary firmware for supported
 Google phones and tablets. Downloads firmware automatically from the
 Google's website.

./recipes-bsp/packagegroups/packagegroup-firmware-dragonboard-apq8074.bb ./recipes-bsp/packagegroups/packagegroup-firmware-dragonboard410c.bb ./recipes-bsp/packagegroups/packagegroup-firmware-dragonboard820c.bb ./recipes-bsp/packagegroups/packagegroup-firmware-dragonboard845c.bb ./recipes-bsp/packagegroups/packagegroup-firmware-ifc6410.bb ./recipes-bsp/packagegroups/packagegroup-firmware-ifc6560.bb ./recipes-bsp/packagegroups/packagegroup-firmware-lenovo-x13s.bb ./recipes-bsp/packagegroups/packagegroup-firmware-rb1.bb ./recipes-bsp/packagegroups/packagegroup-firmware-rb2.bb ./recipes-bsp/packagegroups/packagegroup-firmware-rb3gen2.bb ./recipes-bsp/packagegroups/packagegroup-firmware-rb5.bb ./recipes-bsp/packagegroups/packagegroup-firmware-sm8150-hdk.bb ./recipes-bsp/packagegroups/packagegroup-firmware-sm8350-hdk.bb ./recipes-bsp/packagegroups/packagegroup-firmware-sm8450-hdk.bb ./recipes-bsp/packagegroups/packagegroup-firmware-sm8550-hdk.bb ./recipes-bsp/packagegroups/packagegroup-firmware-sm8650-hdk.bb

packagegroups, selecting necessary firmware and DSP binary packages for
corresponding devices

./recipes-bsp/images/initramfs-firmware-dragonboard820c-image.bb ./recipes-bsp/images/initramfs-firmware-pixel3-image.bb ./recipes-bsp/images/initramfs-firmware-db8074-image.bb ./recipes-bsp/images/initramfs-firmware-sm8550-hdk-image.bb ./recipes-bsp/images/initramfs-firmware-nexus-image.bb ./recipes-bsp/images/initramfs-firmware-rb12-image.bb ./recipes-bsp/images/initramfs-firmware-sm8150-hdk-image.bb ./recipes-bsp/images/initramfs-firmware-ifc6560-image.bb ./recipes-bsp/images/initramfs-firmware-sm8650-hdk-image.bb ./recipes-bsp/images/initramfs-firmware-sm8350-hdk-image.bb ./recipes-bsp/images/initramfs-firmware-sm8450-hdk-image.bb

initramfs images containing firmware to start up remoteprocs and/or Adreno
on corresponding devices

./recipes-bsp/images/initramfs-firmware-image.bb

All-In image pulling firmware for supported RBx and Dragonboard devices.

./recipes-bsp/images/initramfs-firmware-mega-image.bb

All-In firmware image, pulling all packages for all devices, mostly used
for signature validation and (possible) conflict detection

./dynamic-layers/openembedded-layer/recipes-bsp/firmware-woa/firmware-qcom-ecs-liva-qc710_200.0.10.0.bb ./dynamic-layers/openembedded-layer/recipes-bsp/firmware-woa/firmware-qcom-lenovo-miix-630_200.0.6.0.bb ./dynamic-layers/openembedded-layer/recipes-bsp/firmware-woa/firmware-qcom-lenovo-yoga-c630_200.0.19.0.bb ./dynamic-layers/openembedded-layer/recipes-bsp/firmware-woa/firmware-qcom-sc8180x_200.0.111.0.bb ./dynamic-layers/openembedded-layer/recipes-bsp/firmware-woa/firmware-qcom-x1e80100_200.0.32.0.bb

Recipes packaging firmware for several WoA devices by downloading the
binaries community's GitHub repo. Licence status is pretty unclear, as WSUS
archives contain little to no licence information. Downloading from GH repo
can be replaced with accessing WSUS directly, the tool has been implemented
by the UUPDownload project, but it is written in .NET, so it has a huge set
of external dependencies. Ideally the tool should be reimplemented in
Python or in C, so that we can use it to create WoA firmware packages on
laptops themselves (for "normal" distros).

./dynamic-layers/openembedded-layer/recipes-bsp/packagegroups/packagegroup-firmware-ecs-liva-qc710.bb ./dynamic-layers/openembedded-layer/recipes-bsp/packagegroups/packagegroup-firmware-lenovo-miix-630.bb ./dynamic-layers/openembedded-layer/recipes-bsp/packagegroups/packagegroup-firmware-lenovo-yoga-c630.bb ./dynamic-layers/openembedded-layer/recipes-bsp/packagegroups/packagegroup-firmware-sc8180x.bb ./dynamic-layers/openembedded-layer/recipes-bsp/packagegroups/packagegroup-firmware-x1e80100-crd.bb

Packagegroups pulling corresponding firmware images (both from
linux-firmware and from WoA firmware packages).

./dynamic-layers/openembedded-layer/recipes-bsp/images/initramfs-firmware-ecs-liva-qc710-image.bb ./dynamic-layers/openembedded-layer/recipes-bsp/images/initramfs-firmware-lenovo-miix-630-image.bb ./dynamic-layers/openembedded-layer/recipes-bsp/images/initramfs-firmware-lenovo-yoga-c630-image.bb ./dynamic-layers/openembedded-layer/recipes-bsp/images/initramfs-firmware-sc8180x-image.bb ./dynamic-layers/openembedded-layer/recipes-bsp/images/initramfs-firmware-x1e80100-crd-image.bb ./dynamic-layers/openembedded-layer/recipes-bsp/images/initramfs-firmware-mega-image.bbappend

Initramfs images containing these firmware files.

./recipes-kernel/linux-firmware/linux-firmware_%.bbappend

Layer-specific customization to handle conflicts with other

firmware packagages

./recipes-support/initrdscripts/initramfs-module-copy-modules_1.0.bb ./recipes-kernel/packagegroups/packagegroup-qcom-boot.bb ./recipes-kernel/images/initramfs-qcom-image.bb ./recipes-kernel/images/initramfs-rootfs-image.bb

Various pieces of the "jump to the rootfs" lightweight initramfs image,
used by Linaro and Qualcomm developers.

./recipes-support/fastrpc/fastrpc_git.bb ./recipes-support/pd-mapper/pd-mapper_git.bb ./recipes-support/qbootctl/qbootctl_0.1.2.bb ./recipes-support/qrtr/qrtr_git.bb ./recipes-support/rmtfs/rmtfs_git.bb ./recipes-support/tqftpserv/tqftpserv_git.bb

Target Qualcomm-specific tools

./recipes-devtools/debugcc/debugcc_git.bb ./recipes-test/mybw/mybw_git.bb ./recipes-test/bootrr/bootrr_git.bb ./recipes-test/diag/diag_git.bb

Target Qualcomm-specific debugging tools

./recipes-devtools/pil-squasher/pil-squasher_git.bb ./recipes-devtools/qc-image-unpacker/qc-image-unpacker_git.bb ./recipes-devtools/qca-swiss-army-knife/qca-swiss-army-knife_git.bb ./recipes-devtools/qdl/qdl_git.bb ./recipes-devtools/qmic/qmic_git.bb

Native Qualcomm-specific tools

./recipes-kernel/images/esp-qcom-image.bb ./recipes-kernel/images/linux-qcom-uki.bb ./recipes-vendor-in/python/python3-pefile_2024.8.26.bb

UKI support, to be replaced / ported to uki.bbclass

./recipes-devtools/skales/skales_git.bb

Lightweight mkbootimg implementation

./recipes-test/images/initramfs-kerneltest-full-image.bb ./recipes-test/images/initramfs-kerneltest-image.bb ./recipes-test/images/initramfs-test-full-image.bb ./recipes-test/images/initramfs-test-image.bb ./recipes-test/images/initramfs-tiny-image.bb

Several initramfs images useful for testing and development

./recipes-kernel/linux/linux-linaro-qcomlt_6.6.bb

Old 6.6 QcomLT tree, to be removed once qcom-armv7a-modem is ported to
linux-yocto

./recipes-devtools/qtestsign/qtestsign_git.bb ./recipes-bsp/u-boot/u-boot-qcom_git.bb

U-Boot support

./recipes-support/rust-android-sparse/rust-android-sparse_0.6.0.bb

img2simg / simg2img implementation (for handling Google pixel firmware and
for implementing sparse image generation)

./dynamic-layers/openembedded-layer/recipes-devtools/android-tools/android-tools-adbd-cmdline.bb

Layer-specific ADBD customization, used for initramfs-test images

./dynamic-layers/meta-linux-mainline/recipes-kernel/linux/linux-stable_%.bbappend ./dynamic-layers/meta-linux-mainline/recipes-kernel/linux/linux-mainline.bbappend

mainline linux support, used for testing at some point, might be dropped now

./dynamic-layers/openembedded-layer/recipes-graphics/images/qcom-x11-image.bb ./recipes-graphics/mesa/mesa.bb ./recipes-test/lava-test-shell/lava-test-shell.bb

Obsolete, can be dropped ?

./recipes-graphics/xorg-xserver/xserver-xorg_%.bbappend ./recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bbappend

Check if these can be dropped. Might be handled in the upstream

./recipes-graphics/mesa/mesa.bbappend

Platform-specific customization to enable freedreno driver

./recipes-kernel/linux/linux-yocto6.6.bbappend ./recipes-kernel/linux/linux-yocto%.bbappend

Platform-specific linux-yocto implementation

./dynamic-layers/openembedded-layer/recipes-devtools/android-tools/android-tools-conf-configfs_%.bbappend

Platform-specific ADBD customization

./dynamic-layers/openembedded-layer/recipes-navigation/gpsd/gpsd_%.bbappend

Qualcomm-specific GPSd driver, works only on db410c

[ricardosalveti]

• What are the Binary blobs which are needed in meta-qcom, and how do we handle their redistribution.

The non-redistributable binaries would need to be removed (including renesas xhci), should we create a new repository under Linaro for them (e.g. meta-qcom-linaro)?

[lumag]

I think meta-qcom-linaro makes sense. At the same time ideally Qualcomm should try utilizing their power to get a sane licence for that file because, if I remember correctly, it is also required to get that host to work on RB3gen2 (old RB3 had it flashed in the ROM, I don't know if RB3gen2 has the ROM or not).

[ndechesne]

@lumag can you please confirm if the Renesas firmware you have (for the original RB3) is the same as here: https://www.renesas.com/en/products/interface/usb-switches-hubs/upd720201-usb-30-host-controller#documents? That's the one which is recommended to use for RB3 Gen2 (see https://docs.qualcomm.com/bundle/publicresource/topics/80-70015-254/how_to.html#update-usb-and-ethernet-controller-firmware). It is not redistributed by QCOM, but there are instructions for users to download it and flash it.

[ndechesne]

At least based on RB3 and RB3 Gen2 schematics, it's indeed the same Renesas bridge being used on both (uPD720201K8-701-BAC-A)

[lumag]

Yes, it's the same.

My point is that if we distribute rootfs images, they (ideally) should just work without downloading any binaries. Likewise if a user builds rootfs, it should be possible to provide the rom / zip file at the build time. I'm quite insisting here because if a company develops an end-user product using RB3gen2, they should be able to support user products (like providing rootfs, providing updates, etc), without users having to download anything for their device from the 3rd party vendor (Renesas).

[ricardosalveti]

From our investigation there is no ROM in rb3gen2 and we are bound to Renesas terms. It is indeed a pain to manage.

[lumag]

Regarding meta-qcom-linaro. I think instead I will spawn linux-msm/meta-qcom-extras and start moving questionable recipes to that repo. @davidinux @ndechesne @ricardosalveti WDYT?

[ndechesne]

Regarding meta-qcom-linaro. I think instead I will spawn linux-msm/meta-qcom-extras and start moving questionable recipes to that repo. @davidinux @ndechesne @ricardosalveti WDYT?

This is a good idea. I was not sure meta-qcom-linaro was great.. I like your idea much better.

[ricardosalveti]

Yeah, better under linux-msm.

[davidinux]

+1

Sent from Proton Mail for iOS

On Tue, Nov 26, 2024 at 16:48, Ricardo Salveti @.***(mailto:On Tue, Nov 26, 2024 at 16:48, Ricardo Salveti < wrote:

Yeah, better under linux-msm.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

[lumag]

I started the split by preparing to remove Nexus/Pixel and WoA recipes. Are we okay with leaving package-the-binaries recipes for HDKs and db8074 or should I move them to the external repo too?

[lumag]

See #681 , #682 .

[ndechesne]

thanks. Both PRs above look pretty nice cleanups. thanks. Let's keep the HDK and original dragonboard in meta-qcom for now. they are Qualcomm reference designs, so it makes sense. If that becomes a problem in the future we can stage 'legacy' hardware in -extras.

[davidinux]

-extra usually refers to new features not fully baked yet or for which there’s a lesser SLA, whilst -legacy or -obsoleted refers to leftovers that make sense to keep from an historical perspective but are no longer actively maintained nor encourages.

Might be worth considering the -legacy term for dragonboard?

D

Sent from Proton Mail for iOS

On Wed, Nov 27, 2024 at 15:55, Nicolas Dechesne @.***(mailto:On Wed, Nov 27, 2024 at 15:55, Nicolas Dechesne < wrote:

thanks. Both PRs above look pretty nice cleanups. thanks. Let's keep the HDK and original dragonboard in meta-qcom for now. they are Qualcomm reference designs, so it makes sense. If that becomes a problem in the future we can stage 'legacy' hardware in -extras.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

[lumag]

I still have hopes that one day we might have HDK binaries under the 'redistributable' licence. DB8074 and IFC board -- probably no hopes on that side.