Allow TemplateDict to accept Files and expand their content

rickeylev commented 3 months ago

Description of the feature request:

Allow TemplateDict to accept File objects and have it expand them to the content of the file.

This idea comes from wanting to concat files together in an action. Normally this requires run_shell, which poses multi-platform complications.

So instead of this:

plain_zip = <create a zip file>
# bootstrap_template is "#!/bin/bash ...shell code ..."
ctx.actions.expand_template(template=bootstrap_template, output=bootstrap, ...)
ctx.actions.run_shell(
  command = "cat {bootstrap} {zip} > {output}".format(
    bootstrap = bootstrap.path,
    zip = plain_zip.path
    output = executable_zip.path
  )
)

One could do this:

computed = TemplateDict()
computed.add_file("%ZIP%", plain_zip)
ctx.actions.expand_template(
  # execeutable_zip_template is "#!/bin/bash ...shell..\n%ZIP%"
  template=executable_zip_template,
  output=executable_zip.path,
  computed_substitutions = computed
)

Similarly, this would come in handy for a two-stage bootstrap I need to create: stage one is some shell code that computes some values to feed into the second stage python file. Basically a template that looks like

#!bin/bash
foo="%something%"
bar=<computed value>
file=<python program with $bar in it>
exec $file

Having it all in one file is sort of annoying. Being able to split it in two would be handy.

Which category does this issue belong to?

Rules API

What underlying problem are you trying to solve with this feature?

Being able to use file content in template actions

Which operating system are you running Bazel on?

linux

What is the output of `bazel info release`?

7.x

If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

No response

What's the output of `git remote get-url origin; git rev-parse HEAD` ?

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

comius commented 2 months ago

cc @hvadehra for triage

hvadehra commented 2 months ago

I'd be happy to review a PR that implements this.

It's been a while so my memory is a little hazy on the details, but some potential things to watch out for are:

The template expansion code (as currently written) assumes all substitution values can be evaluated at analysis time, which this change would violate. In particular, the action fingerprinting will need to be adapted.
The error surface would expand, and this will likely interact poorly with the class hierarchy.
TBD: what would/should aquery print for such an action?

bazelbuild / bazel