cmd/cue: print the files that would be read by a command

[jpluscplusm]

Is your feature request related to a problem? Please describe.

cue cmd tooling is great, but AFAICT it currently lacks a (built-in) feature: the ability to selectively recreate files based on external criteria. (This issue is not asking for the tooling to gain that ability!)

For situations where recreation of some but not all files is desirable (perhaps due to lengthy network calls, costly CUE evaluations, etc), I fall back to using make to drive my cue invocations, as it has first-class support for at least one kind of external criteria - file modification times. That's not perfect -- I wish make had first-class support for other kinds of externality e.g. HTTP responses -- but it suffices for many situations.

To get make to recreate only certain files (or perform certain actions) I need to teach it which files' timestamps it needs to check. Either that, or I need to tell it that it should always perform certain actions which kind of negates the point of using make over cue cmd, modulo make's significantly more terse syntax.

So when I'm using make to drive cue in any kind of package-based situation, I'm faced with a problem: which files do I teach make it should examine? I outline a few options I've used in Describe alternatives you've considered, below, but none is great. I'd like cue to help me with this.

Describe the solution you'd like

I'd like there to be a (relatively) globally-available option, --just-show-me-the-files, that prints the filenames that any particular cue invocation would read and evaluate.

I'd like it to be quick (i.e. not actually performing the CUE evaluation, but only reading and traversing the package- and import-preamble of each CUE file that's in scope) so it can be used in make situations like this (where the command would be called on every make invocation, during the DAG assembly phase):

rebuild: lengthy_rebuild_output.txt

lengthy_rebuild_output.txt: rebuild-input.json
        some-lengthy-rebuild-command rebuild-input.json >lengthy_rebuild_output.txt

rebuild-input.json: $(shell cue export .:rebuild_input --just-show-me-the-files)
        cue export .:rebuild >rebuild-input.json

This would allow a make rebuild to be called with impunity, only taking action when required.

I'd like the option to be available for as broad a set of cue subcomands as makes sense, but export would seem to be the irreducible set that /needs/ this option (IMHO).

I'd like the option to be valid when placed at the end of a command (after the package/file scope is specified), so that the example above could actually be expressed as:

# ... targets as above

CMD_REBUILD_INPUT:=cue export .:rebuild_input
rebuild-input.json: $(shell $(CMD_REBUILD_INPUT) --show-me-the-files)
        $(CMD_REBUILD_INPUT) >rebuild-input.json

Describe alternatives you've considered

1. Always rebuild:

   .PHONY: rebuild-input.json
   rebuild-input.json:
          cue export .:rebuild >rebuild-input.json

   Pros: easy to implement.
   Cons: slow rebuild cycle.

2. Pessimistic find:

   rebuild-input.json: $(shell find . -type f -name '*.cue')
          cue export .:rebuild >rebuild-input.json

   Pros: still easy to implement.
   Cons: Any CUE file being updated, even if not in scope, causes a slow rebuild

3. Manually-maintained dependency list:

   rebuild-input.json: file1.cue file2.cue $(wildcard cue.mod/pkg/example.com/*.cue) $(wildcard local_package/*.cue)
          cue export .:rebuild >rebuild-input.json

   Pros: accurate; as minimal a set of dependencies as can be described. Cons: painful to implement and update; implies knowledge of internal and external package layouts

4. grep-for-package-foo:

   I can't find an example of where I've done this, but it involves recursively grepping for ^package <foo>$ and emitting only the filename. It obviously doesn't work for situations where more than one CUE package is involved, or if there are transitive imports.

Additional context

I recognise this isn't solving a strictly-CUE-related problem, but I believe cue's co-existence alongside Make (and Make-alike tooling that would also be improved by understanding any specific cue invocation's internal dependency graph) is a reasonable medium-term certainty.

cue cmd may evolve to replace any and all external task runners but, until that happens, giving that wide ecosystem of tools a way to introspect what CUE would do seems like a useful interoperability olive branch of peace :-)

I believe the flag's existence would also help in some other situations e.g. automated CI processes that could record and track the number of files being processed at different stages of a build pipeline, over time, giving operators insight into material perf deltas and correlating their associated commits.

[myitcv]

What you are looking for here is cue list. See the docs for the ~equivalent go list.