Share dynamic values across dynamic_output actions

[aherrmann]

Adds support for sharing values across dynamic_output actions.

The function given to a dynamic output action can now return a list of providers. The ctx.actions.dynamic_output function returns a handle to the future values of these providers. Another dynamic output action can take these handles in, which allows the function to resolve these handles into their resolved values. This way it is possible to share values that are obtained within a dynamic output lambda with another dynamic output lambda.

Includes a small example demo

$ cd examples/dynamic_value
$ cargo run --bin buck2 build //:target_b

The motivation comes from Haskell builds, where we want to achieve cross package module granular builds as described in https://github.com/tweag/rules_haskell/issues/1631. In that case we need to know the module dependency graph across package boundaries and track a tset for each module that captures its transitive module dependencies across package boundaries. Without this feature this is not possible without piping these module dependency graphs through the filesystem and reloading and reanalyzing them over and over again in each packaeg's dynamic output action.

• Example using dynamic values
• Implement dynamic values

[aherrmann]

We have now successfully applied these changes on a Haskell project to achieve cross package module granular incremental builds. The dynamic value introduced here works to propagate module dependency graphs across package boundaries through tsets.

I'd like to open this up for review and discuss whether the API change is acceptable. This adds an example to test the feature. cc @cjhopman

[stepancheg]

Assigning the PR to myself.

However, this week we have internal conference, and next week I'm on holidays. Can it wait two weeks @aherrmann?

[aherrmann]

Hi @stepancheg, thank you for taking this on! Yes, of course, no problem if this waits two weeks. :slightly_smiling_face:

[aherrmann]

@stepancheg I hope you had a good conference and vacation! Did you have a chance to take a look at this since you're back?

[stepancheg]

@aherrmann working on it. Sorry for delay!

[aherrmann]

No worries, I can relate. Thank you for looking into it!