cquery missing dependencies or incorrectly reporting transitions

[illicitonion]

Description of the bug:

I am running cquery to find out, including transitions, about the dependency chain of configured targets of a go_proto_library.

If I bazel cquery --output=jsonproto --transitions=lite 'deps(//:epb_go_proto)' some go_proto_library, configuredRuleInputs declares there's a dependency on @io_bazel_rules_go//:go_context_data in the same configuration as the target itself.

That target itself claims to have a dependency on @io_bazel_rules_go//:stdlib also in the same configuration.

However, there is no entry in the cquery output for @io_bazel_rules_go//:stdlib in that configuration. That target is present in three configurations, but not the one declared.

I would expect the transition following and expansion of cquery to ensure any ConfiguredTarget which appears in configuredRuleInputs is also present in the output of a deps query.

From inspecting the relevant rules, go_proto_library just has a regular dependency on @io_bazel_rules_go//:go_context_data, and then @io_bazel_rules_go//:go_context_data self-transitions.

As far as I can tell, these dependencies don't go via any toolchains, and though go_proto_library does have an aspect, that's not relevant for this dependency path.

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

With this WORKSPACE:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "io_bazel_rules_go",
    sha256 = "6b65cb7917b4d1709f9410ffe00ecf3e160edf674b78c54a894471320862184f",
    urls = [
        "https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.39.0/rules_go-v0.39.0.zip",
        "https://github.com/bazelbuild/rules_go/releases/download/v0.39.0/rules_go-v0.39.0.zip",
    ],
)

load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies")

go_rules_dependencies()

go_register_toolchains(version = "1.19.3")

http_archive(
    name = "com_google_protobuf",
    sha256 = "c6003e1d2e7fefa78a3039f19f383b4f3a61e81be8c19356f85b6461998ad3db",
    strip_prefix = "protobuf-3.17.3",
    urls = [
        "https://github.com/protocolbuffers/protobuf/archive/v3.17.3.tar.gz",
    ],
)

load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")

protobuf_deps()

and this BUILD file:

load("@rules_proto//proto:defs.bzl", "proto_library")
load("@io_bazel_rules_go//go:def.bzl", "go_binary", "go_library")
load("@io_bazel_rules_go//proto:def.bzl", "go_proto_library")

proto_library(
    name = "epb_proto",
    srcs = ["example.proto"],
    visibility = ["//visibility:public"],
)

go_proto_library(
    name = "epb_go_proto",
    importpath = "github.com/example/project/epb",
    proto = ":epb_proto",
    visibility = ["//visibility:public"],
)

go_library(
    name = "project_lib",
    srcs = ["main.go"],
    importpath = "github.com/example/project",
    visibility = ["//visibility:private"],
    deps = [":epb_go_proto"],
)

go_binary(
    name = "project",
    embed = [":project_lib"],
    visibility = ["//visibility:public"],
)

Run bazel cquery --output=jsonproto --transitions=lite 'deps(//:epb_go_proto)'

And inspect the output for the particular targets I've mentioned in the issue description.

Which operating system are you running Bazel on?

macOS

What is the output of bazel info release?

release 6.1.1

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

[illicitonion]

/cc @gregestren @fmeum @sdtwigg

[katre]

@gregestren Please triage this, it may be related to your work on cquery and aspects.

[gregestren]

I generally think cquery accuracy issues are from aspects or toolchains (https://github.com/bazelbuild/bazel/issues/16310) but @illicitonion mentions neither are relevant here.

To be clear, do you think this is specifically an issue with --output=jsonproto? Does this look similar with default output formatting?

[illicitonion]

[Edited - I was looking at an incorrect target]

No, it doesn't look specific to the output format, exactly the same is present in the default output format

//:epb_go_proto (7dbfde3)
  proto#//:epb_proto#(Starlark transition:/private/var/tmp/_bazel_dwagnerhall/2cc5552d5ea0ab87cc9070d7832c1463/external/io_bazel_rules_go/go/private/rules/transition.bzl:255:36 + (TestTrimmingTransition + ConfigFeatureFlagTaggedTrimmingTransition)) -> 7dbfde3
  compilers#@io_bazel_rules_go//proto:go_proto#(TestTrimmingTransition + ConfigFeatureFlagTaggedTrimmingTransition) -> 7dbfde3
  $go_context_data#@io_bazel_rules_go//:go_context_data#(Starlark transition:/private/var/tmp/_bazel_dwagnerhall/2cc5552d5ea0ab87cc9070d7832c1463/external/io_bazel_rules_go/go/private/rules/transition.bzl:175:37 + (TestTrimmingTransition + ConfigFeatureFlagTaggedTrimmingTransition)) -> 7f452ad
  $allowlist_function_transition#@bazel_tools//tools/allowlists/function_transition_allowlist:function_transition_allowlist#NullTransition -> 
  [toolchain dependency]#@go_sdk//:go_darwin_amd64-impl#(TestTrimmingTransition + ConfigFeatureFlagTaggedTrimmingTransition) -> 7dbfde3

Following $go_context_data#@io_bazel_rules_go//:go_context_data#(Starlark transition:/private/var/tmp/_bazel_dwagnerhall/2cc5552d5ea0ab87cc9070d7832c1463/external/io_bazel_rules_go/go/private/rules/transition.bzl:175:37 + (TestTrimmingTransition + ConfigFeatureFlagTaggedTrimmingTransition)) -> 7f452ad I see:

Starlark transition:/private/var/tmp/_bazel_dwagnerhall/2cc5552d5ea0ab87cc9070d7832c1463/external/io_bazel_rules_go/go/private/rules/transition.bzl:175:37 -> @io_bazel_rules_go//:go_context_data (7f452ad)
  cgo_context_data#@io_bazel_rules_go//:cgo_context_data#(TestTrimmingTransition + ConfigFeatureFlagTaggedTrimmingTransition) -> 7f452ad
  coverdata#@io_bazel_rules_go//go/tools/coverdata:coverdata#(TestTrimmingTransition + ConfigFeatureFlagTaggedTrimmingTransition) -> 7f452ad
  go_config#@io_bazel_rules_go//:go_config#(TestTrimmingTransition + ConfigFeatureFlagTaggedTrimmingTransition) -> 7f452ad
  nogo#@io_bazel_rules_nogo//:nogo#(exec + (NoTransition + ConfigFeatureFlagTaggedTrimmingTransition)) -> a22f584
  stdlib#@io_bazel_rules_go//:stdlib#(Starlark transition:/private/var/tmp/_bazel_dwagnerhall/2cc5552d5ea0ab87cc9070d7832c1463/external/io_bazel_rules_go/go/private/rules/transition.bzl:277:34 + (TestTrimmingTransition + ConfigFeatureFlagTaggedTrimmingTransition)) -> 7f452ad
  $allowlist_function_transition#@bazel_tools//tools/allowlists/function_transition_allowlist:function_transition_allowlist#NullTransition -> 
  [toolchain dependency]#@go_sdk//:go_darwin_amd64-impl#(TestTrimmingTransition + ConfigFeatureFlagTaggedTrimmingTransition) -> 7f452ad

[gregestren]

Thanks. Got to do some more diagnosing then... Any non-format-specific accuracy issue needs to be understood.

[gregestren]

I can't resolve https://artifacts.apple.com/github.com/protocolbuffers/protobuf/archive/v3.17.3.tar.gz. Am I missing something obvious?

[illicitonion]

Apologies, I've edited the original post to remove the address of a private mirror of https://github.com/protocolbuffers/protobuf/archive/v3.17.3.tar.gz

[gregestren]

Phew. I thought I was failing some basic repo mapping test. :)

[illicitonion]

It looks like it was been fixed by 9575c7fa86458717c154f08ff8a7f7aff000f6ae 🎉

That change looks very hard to backport to 6.4.0, as it has a lot of dependencies, but if anyone has any ideas for how to do so, it'd be really handy to have done!

[gregestren]

I agree that will be hard to backport. But I still don't understand why that level of refactoring would be necessary for a bug that manifests in all output formats.

[illicitonion]

My suspicion from starting to debug this myself is that the manual implementation following split transitions wasn't also applying self transitions on the transition target, and that possibly something in that change started using more general transition-applying code which applied more of the relevant transitions, but I didn't delve super deep into the actual fix.