search

facebook / buck2

Build system, successor to Buck
https://buck2.build/
Apache License 2.0
3.53k stars 215 forks source link

Add a bxl for generating a unified compile_commands. #510

Open zjturner opened 10 months ago

zjturner commented 10 months ago

This adds a simple bxl for generating a global compilation database for all targets in a build. It is very fast, able to generate a database for around 4,000 files in under 4 seconds.

This opens the door for external users being able to integrate with traditional LSPs like clangd.

This addresses several outstanding issues for external users, including #194, #307, and #452.

zjturner commented 10 months ago

@Overhatted

zjturner commented 10 months ago

I'm not sure why this failed, but it doesn't appear to be related to anything in my PR. @ndmitchell do you have any suggestions for how I can fix this?

JakobDegen commented 10 months ago

A fix for the CI issue landed on master, you should pick it up if you rebase.

I have to say I don't quite understand the need for aquery here. The obvious thing to me would have been to run analysis on the whole world to gather all the compilation DBs, and then just merge them. How does that approach go wrong?

zjturner commented 10 months ago

A fix for the CI issue landed on master, you should pick it up if you rebase.

I have to say I don't quite understand the need for aquery here. The obvious thing to me would have been to run analysis on the whole world to gather all the compilation DBs, and then just merge them. How does that approach go wrong?

When I tried that (maybe I did it wrong) it was extremely slow. It was an early attempt, so my memory is rusty, but I recall that generation of all the compilation dbs was very slow, because the sub target to generate the compilation db in the first place had dependencies on other targets (didn’t investigate what) and it would build things before it was able to generate the compilation db. I don’t know what actions that “build things” step ran (didn’t investigate), I just remember that it was slow. This method generates a universe compilation db very fast.

JakobDegen commented 10 months ago

because the sub target to generate the compilation db in the first place had dependencies on other targets

The only action that's run by compilation DB generation does argsfile expansion. I could maybe imagine this forcing targets in the srcs attribute or something of that sort to get built, but we should fix that if so. But if you don't need argsfile expansion, which seems to be the case here, then I think just having adding a compilation-database-unexpanded subtarget, and combining that across the entire repo, would be the right thing to do

ndmitchell commented 9 months ago

So the context here is that when you generate a compilation database, you basically have two choices:

  1. Generate the database with zero dependencies. That basically ends up looking like the aquery solution.
  2. Generate the database with lots of dependencies - this ensures that things like generated code are produced properly. It also means that you can literally run the command lines produced and they work.

I believe our [compilation-database] target goes for lots of dependencies. I originally worked on a version with zero dependencies, but then our IDE teams wanted them added to support their use cases.

@JakobDegen is suggesting we have [compilation-database-unexpanded]. We probably should, but I don't think this solution is harmful, so suggest we merge it anyway?

cbarrete commented 1 month ago

@zjturner Did you plan on finishing this PR and getting it merged? If not, would you be ok with someone picking it up? It seems that @ndmitchell only had minor comments about this version and was otherwise ready to merge it.