Add BitBake (Yocto) as supported package manager

[tsteenbe]

Add support for the BitBake (Yocto) dependency manager to the analyzer module.

Links:

BitBake manual: https://www.yoctoproject.org/docs/1.6/bitbake-user-manual/bitbake-user-manual.html Useful Bitbake commands: https://community.nxp.com/docs/DOC-94953

• bitbake-layers show-layers
• bitbake-layers show-recipes

Bitbake repos: https://git.yoctoproject.org/ Projects starting with poky- and meta- likely contain bitbake files

[pattivacek]

I recommend using the latest version of the bitbake manual: https://www.yoctoproject.org/docs/current/bitbake-user-manual/bitbake-user-manual.html. 1.6 is pretty old at this point.

[sschuberth]

Maybe also have a look at https://github.com/Waldleufer/archproj-bmwteam of which our contributor @andreas-bauer has a fork.

[sschuberth]

FYI, there's been work started in the bitbake branch.

[tsteenbe]

The questions below outline the prerequisites of things we need to figure out prior to be able to add Yocto support to ORT

1. How can you detect a project uses this specific package manager?
2. How can you get the declared license for a package?
3. How to can one get dependency tree including package names, versions?
4. How can one obtain the source code for a dependency? (e.g. the source code repository + revision or the sources artifact URL)
5. How can one separate code dependencies from build/test ones?
6. Can you provide example projects that can be used to test the implementation?

[pattivacek]

I think it's still an open question if it's really appropriate the use ORT on Yocto in the first place. In our case, we provide meta-updater and some additional supporting layers as open-source projects to help our users integrate aktualizr into their projects. However, meta-updater (and essentially all of Yocto) is just metadata. We don't normally deploy or deliver the software built with Yocto, just the metadata that explains how to build it.

The metadata itself that we use is released under friendly open-source terms. The software built with the metadata is more complex, and Yocto has some internal tooling for detecting it. I wrote a script that can collect the licenses for a given package and its dependencies and another script that uses the first to specifically find the dependencies of aktualizr. However, I'm not sure anymore if that is actually useful.

So before those six questions can be answered, we need to clarify: are we trying to search for licenses of the metadata that is Yocto or the underlying software that is built with Yocto?

[sschuberth]

are we trying to search for licenses of the metadata that is Yocto or the underlying software that is built with Yocto?

The latter, IMO. Or more generically: We need to determine the licenses of what is being distributed in the end.

[pattivacek]

We need to determine the licenses of what is being distributed in the end.

Even though we aren't distributing the end product ourselves? Our users normally customize their images with their own software and build them themselves. We don't deliver images as a product, but rather just the metadata to help include our software in their images.

[sschuberth]

Let's leave the term "(end-)product" aside for a moment. It does not matter which kind of software you distribute (e.g. whether it's a product, SDK, or some tools that build the actual "product"), but if you distribute software (esp. in binary form), you need to take care to fulfill all obligations of the licenses of the software you are distributing.

So if you as a company ship a .tar.gz archive to your customers that contains software (or metadata) that allows your customers to build their own binaries of something, then you still need to gather the licenses and assemble a disclosure document for the software (or metadata) in the .tar.gz archive. I hope that makes it clearer.

[mmurto]

5. How can one separate code dependencies from build/test ones?

We're currently trying to approach this with file level, rather than package level license granularity, so it might not be completely relevant for ORT, but here's where we're at:

Yocto has some built in tooling to scan the built image for source files used to build it. We're using this to get the hash values to find licenses of individual files, but it the tooling might be useful for separating the code in the build from build/test ones for you aswell. We've described the process in a little more detail here (heavily WIP).

[pattivacek]

It does not matter which kind of software you distribute (e.g. whether it's a product, SDK, or some tools that build the actual "product"), but if you distribute software (esp. in binary form), you need to take care to fulfill all obligations of the licenses of the software you are distributing.

Exactly. We only distribute one project's source code (which is C++, so we can scan it directly instead of going through Yocto) and the rest is just metadata. We can scan the metadata, but in our case, I don't believe it makes sense to scan the software described by the metadata. We don't distribute that software, just (some of) the metadata.

That said, that's just our use case. I don't know if there are others using Yocto to build images that then get distributed, but we (the Edge OTA team) do not do that.

[sschuberth]

For reference, I just learned that besides Double Open's original https://github.com/doubleopen-project/meta-doubleopen, there's also https://github.com/fossas/meta-fossa that might be worth looking at.

[tsteenbe]

Also https://github.com/fosslight/fosslight_yocto_scanner may be worth looking into..