perf: reduces bazel query output

[iamricard]

I spent a little bit of time digging into why this extension kept crashing in our codebase. After some investigation, I noticed the bazel query helper requests the output as protobufs. The output is very useful for tools, but it does come with a few default options that bloat the query's output:

• --proto:locations=true: Whether to output location information in proto output at all.
• --proto:output_rule_attrs="all": Comma separated list of attributes to include in output.
• --proto:rule_inputs_and_outputs=true: Whether or not to populate the rule_input and rule_output fields.
• --proto:default_values=true: If true, attributes whose value is not explicitly specified in the BUILD file are included; otherwise they are omitted.

I dug through the codebase and it seems like none of the values populated are used anywhere in the extension. For smaller projects this is probably fine. For the codebases we (DataDog) work with, the in-memory representation of the entire graph is far too large. One of our repos has close to 900k targets. Ran a small benchmark script to capture how much memory processing the output takes before and after. Ran with: time node --max-old-space-size=20000 bench.mjs:

• with default flags
  • before_all=6.6668701171875mb
  • after_exec_sync=54.42418670654297mb
  • after_decode=17840.955001831055mb
  • node --max-old-space-size=20000 test.mjs 63.65s user 36.90s system 89% cpu 1:51.96 total
• with trimmed outputs
  • before_all=6.664710998535156mb
  • after_exec_sync=11.035118103027344mb
  • after_decode=1131.992919921875mb
  • node --max-old-space-size=20000 test.mjs 2.29s user 1.05s system 37% cpu 9.024 total

Benchmark codebase:

// <large-repo>/bench.mjs
import protos from "/home/user/.vscode/extensions/bazelbuild.vscode-bazel-0.7.0/out/src/protos/protos.js";
import { exec } from "node:child_process";

const before_all = process.memoryUsage().heapUsed / 1024 / 1024;
const go = new Promise((resolve, reject) => exec(
    // swapped with the new query
    "bzl query \"kind('.* rule', ...)\" --output proto",
    { cwd: "/Users/ricard.sole/dd/dd-source", encoding: null, maxBuffer: Number.MAX_SAFE_INTEGER },
    (error, stdout, _) => {
        if (error) {
            reject(null);
        } else {
            const after_exec_sync = process.memoryUsage().heapUsed / 1024 / 1024;
            protos.blaze_query.QueryResult.decode(stdout);
            const after_decode = process.memoryUsage().heapUsed / 1024 / 1024;
            console.log(`\
* before_all=${before_all}
* after_exec_sync=${after_exec_sync}
* after_decode=${after_decode}`);
            resolve(stdout);
        }
    },
));

await go;

Should fix bazelbuild/vscode-bazel#296 in most setups. Ideally there would be a longer-term rework to lazily load and process the query output but that's quite a large undertaking.

[iamricard]

CI just seems to be kind of busted 👀

SyntaxError: Unexpected token '??='
--
  | at wrapSafe (internal/modules/cjs/loader.js:1029:16)
  | at Module._compile (internal/modules/cjs/loader.js:1078:27)
  | at Object.Module._extensions..js (internal/modules/cjs/loader.js:1143:10)
  | at Module.load (internal/modules/cjs/loader.js:979:32)
  | at Function.Module._load (internal/modules/cjs/loader.js:819:12)
  | at Module.require (internal/modules/cjs/loader.js:1003:19)
  | at require (internal/modules/cjs/helpers.js:107:18)
  | at Object.<anonymous> (/usr/lib/node_modules/npm/node_modules/@npmcli/agent/lib/index.js:7:15)
  | at Module._compile (internal/modules/cjs/loader.js:1114:14)
  | at Object.Module._extensions..js (internal/modules/cjs/loader.js:1143:10)

[iamricard]

Thanks, LGTM! #312 should fix the CI failures. Once that merges, can you rebase on top?

yup sounds good!

[iamricard]

Thanks, LGTM! #312 should fix the CI failures. Once that merges, can you rebase on top?

done!

[iamricard]

thanks for merging @coeuvre 🙇🏼 do you have an idea of when y'all are likely to cut a new release? i'd like to let folks know at datadog they can go back to using the published extension 🎉

[coeuvre]

I am not sure. @jfirebaugh has been managing the releases. John, what do you think?

[daivinhtran]

@iamricard I'm afraid this optimization is breaking CodeLens (or maybe only on the bazel version my repo is using).

I tried reverting the proto flags and CodeLens worked as normal. See the screencast when the flags are used. A few obvious bugs:

• Clicking on the targets in Bazel Build Targets no longer works.
• CodeLens feature that allows navigating to a target no longer works.
• The tooltips for build command now all shows at the top of the BUILD file instead of being inline with the targets.