Information about where third party libraries are from should be present in the build graph

[shs96c]

Feature request

When using a go_repository, the repository rule knows the origin and precise version information of the third party repo being loaded. However, because this information is not encoded anywhere in the generated targets, it is impossible to retrieve this information in a build rule or aspect. This makes it impossible to walk the dependency graph of a target and find out which versions of which dependencies were used.

In contrast, rules_jvm_external embeds this information in the tags of generated rules.

Having this information available makes it simpler to generate artifacts such as SBOMs (Software Bill of Materials) which can be consumed by other tooling.

What version of gazelle are you using?

0.26.0

What version of rules_go are you using?

Snapshot of cde7d7bc27a34547c014369790ddaa95b932d08d

What version of Bazel are you using?

5.2.0

Does this issue reproduce with the latest releases of all the above?

Yes

What operating system and processor architecture are you using?

N/A

[sluongng]

Revisiting this topic today, I realize that we could start creating a Gazelle extension for package_info run rules_license https://github.com/bazelbuild/rules_license/blob/main/rules/package_info.bzl#L73.

This is essentially a copy/paste of the language/bazel/visibility extension we have today.

Then before each go_repository runs Gazelle, we could append the appropriate directive with version information into the BUILD file. https://github.com/bazelbuild/bazel-gazelle/blob/a1be47403b1aa95dd8f56b6a4954095dd1f572f1/internal/go_repository.bzl#L274-L278 So that gazelle could generate other BUILD files with relevant package_info targets.

We probably need something that detects whether rules_license does exist in the WORKSPACE to conditionally add the rule though.

[sluongng]

@fmeum @linzhp what do you think about Gazelle depending on rules_license directly to make this happen?

https://github.com/bazelbuild/bazel/discussions/18473 is being discussed to make the dependency optional. But I think there should be relatively little harm for go_repository to depend on rules_license in order to enable better supply chain control.

[fmeum]

I don't see a problem with Gazelle depending on rules_license, both in WORKSPACE and MODULE.bazel land. A Gazelle extension sounds like exactly the right way to handle this with the least amount of additional complexity.

[achew22]

Personally I would like to avoid pushing any additional plugins into this repo. To me, rules_license is the most obvious place to have this code live, like we do for skylib. Anyone who has rules_license already installed should have a very easy time adding a single line to their gazelle binary and adding this in

[sluongng]

cc: @aiuto

I think introducing a Gazelle extension into rules_license code base, primarily starlark and python, would require a Golang reviewer. From the conversation in the Bazel's discussion, it seems like the current goal is to keep rules_license as lean/small as possible. So it's faster and more correct to add it here, not as part of Gazelle but of go_repository.

We also might be interested to couple the dependency with the existing language/bazel/visibility extension to include the version data inside package() metadata attribute.

Lastly, we might be interested in tightly coupling this with go_repository starlark to add the correct directive. Subsequently, we would want to adjust rules_go and may be language/go extension to start consuming the versioning information to create buildID similar to go build ..

[darkrift]

Ping, any plans to get this move forward ?

[aiuto]

The rollout of bzlmod changed the plans a bit. In 2023, I thought it was feasible to inject package_info targets into BUILD files generated by the individual repository rules. While that is possible (and allows graph analysis during build time), the more cost effective solution is to build a graph down to repositories, and then merge with version information from the lock files to produce an SBOM (or other audit). We're doing something like that for Bazel itself, using the maven lock file. That was working before bzlmod but broke with a lock file change at bazel 7.x.

As far as staffing. I or the someone in the Bazel team will be working on reading from the lock files (bzlmod first, then maven). We are not currently staffing package_info() injection into generated BUILD files for the other repository rules (go, python, foreign_cc, nodejs, ...). Any help for the language specific rules would be appreciated. That can be done in those repositories, without any change in rules_license. I'll be happy to review those PRs (even I I can not be the ultimate approver).

[zecke]

The rollout of bzlmod changed the plans a bit. In 2023, I thought it was feasible to inject package_info targets into BUILD files generated by the individual repository rules. While that is possible (and allows graph analysis during build time), the more cost effective solution is to build a graph down to repositories, and then merge with version information from the lock files to produce an SBOM (or other audit). We're doing something like that for Bazel itself, using the maven lock file. That was working before bzlmod but broke with a lock file change at bazel 7.x.

As far as staffing. I or the someone in the Bazel team will be working on reading from the lock files (bzlmod first, then maven). We are not currently staffing package_info() injection into generated BUILD files for the other repository rules (go, python, foreign_cc, nodejs, ...). Any help for the language specific rules would be appreciated. That can be done in those repositories, without any change in rules_license. I'll be happy to review those PRs (even I I can not be the ultimate approver).

Do you have a pointer to the design document? I am interested in forms of SBOM generation for Go and JS/TS.

[aiuto]

The best backgrounders are the ones that the rules_license landing page points to. https://github.com/bazelbuild/rules_license

I am interested in forms of SBOM generation for Go and JS/TS.

You should think about SBOM generation as completely decoupled from the language you are using. Eventually, you should be able to produce a graph that says I compiled JS with this instance of a compiler, which itself is written in C++.

That said, since you asked about design docs and Go specifically, I can assure you that no one has written a design about how bazel-gazelle and or rules_go will fix their importer so that the generated BUILD files include package_info() and or license() declarations. We need volunteers from the Go community to design and build that. Once that happens, those drive the SBOM content. The same holds true for all of the languages where the Bazel team does not directly own the import rules.