rogpeppe / go-internal

Selected Go-internal packages factored out from the standard library
BSD 3-Clause "New" or "Revised" License
823 stars 67 forks source link

Feature proposal: use environment variables as custom conditions in a txtar script #241

Closed rudifa closed 3 months ago

rudifa commented 6 months ago

In a txtar script I want to be able to use environment variables to condition the test step execution, like this

# demo the proposed 'envvar as custom condition' feature

[unix] exec echo "hello unix"
[MYVAR] exec echo "hello MYVAR"
[!MYVAR] exec echo "bye MYVAR"

Motivation:

In a script like this one I can collect a number of related test cases (a cue input, a current possibly buggy output of cue fmt, a dump of the ast, e.t.c.):

#issue2567.txtar

exec cue-ast-print 2567.cue
cmp stdout 2567.ast
exec cue fmt 2567.cue
cmp 2567.cue 2567.got.cue

exec cue-ast-print 2567.0.cue
cmp stdout 2567.0.ast
exec cue fmt 2567.0.cue
cmp 2567.0.cue 2567.0.got.cue
...

Now I want to prefix some of the test steps with custom conditions

#issue2567.txtar

[DOPRINT] exec cue-ast-print 2567.cue
[DOPRINT] cmp stdout 2567.ast
exec cue fmt 2567.cue
[DOCMP] cmp 2567.cue 2567.got.cue
[DOVET] exec cue vet 2567.cue

[DOPRINT] exec cue-ast-print 2567.0.cue
[DOPRINT] cmp stdout 2567.0.ast
exec cue fmt 2567.0.cue
[DOCMP] cmp 2567.0.cue 2567.0.got.cue
[DOVET] exec cue vet 2567.0.cue
...

This would let me run the same script at different times with different purposes, while having similar test cases with their related data (in, got, ast, ...) all in one script, e.g.:

% DOPRINT=1 testscript issue2567.txtar  # compare the ast printout to the previously updated values
% DOCMP=1 testscript -u issue2567.txtar # update the expected outputs to the current values
% DOCMP=1 DOVET=1 testscript issue2567.txtar # compare output to expected and vet the output

The new feature would be documented like this:

The command prefix [cond] indicates that the command on the rest of the line
should only run when the condition is satisfied. The predefined conditions are:

  - [short] for testing.Short()
...

In addition, environmnent variables can be used as custom conditions, for example

  - [MYVAR] <command>

such that in `MYVAR=1 testscript <scriptfile>` the <command> is executed, 
while in `testscript <scriptfile>` the <command> execution is skipped.

Comments on the proposed feature are invited.

mvdan commented 3 months ago

Sure - you can already do this yourself by adding custom conditions. I don't believe this needs to be part of testscript itself, and I think users might want different behavior in terms of how to convert an env var string to a boolean. Some users might want to support just 1, others https://pkg.go.dev/strconv#ParseBool, others "is not empty", or others even "is set" like in https://pkg.go.dev/os#LookupEnv.

I agree that it can be useful, but it's pretty easy to write your own conditional function, and you then get to choose exactly what boolean logic you want, so I think that's the best approach.