[Proposal] Preprocessing: directives, response files and environment

[KathleenDollard]

One of our design challenges is handling directives. The .NET libraries will focus on Posix and Windows standards and expectations and directives are unique to System.CommandLine. They also solve an interesting problem: how can a CLI parser add default behavior without breaking existing CLI designs? This was important for the current tab completion implementation and diagramming, for example and is considered essential by the .NET CLI folks.

Directives fall into two categories: those that set a value for later use (similar to a normal option) and those that perform an action (similar to a command or an option with a behavior like --help).

You can currently set an environment variable via a directive, and that variable will live for the life of your application.

We have had requests to allow defaults to be easily retrieved from environment variables.

Supporting response files is very convenient for users, and important in some scenarios. Unfortunately, there is not single standard for their shape, so including them in the parser layer is problematic.

Directive support including setting environment variables, diagramming and suggest would be significant breaking changes if not supported. Similarly, not supporting response files would be a significant breaking change. Thus, we need to find a design.

This proposal solves these problems via preprocessing with minimal changes to the parser. This simple approach has drawbacks and there are other approaches that could be used.

A simple implementation of preprocessing

A simple approach to preprocessing would be:

• Change CliSubsystem.Intialize to accept the raw input args, and allow add a property to the configuration/rootcommand/parser for how many args to skip, possibly called SkipArgs
  • Subsystem.Initialize would do its own parse and removed what it handled. For directives, they appear first with an open/close of [] so would be fairly easy to parse, setting the SkipArgs value.
• This approach would be constrained to acting at the beginning of the args array

A benefit of this approach is low concept count. It's just another way to use subsystems.

Consideration is needed for errors in Initialize methods.

Early exit

It may also be helpful to allow early termination, such as a subsystem that checked for elevated access or a license prior to parsing. We just need a use case that shows this preferable to checking before calling System.CommandLine.

This would occur if the SkipArgs was greater than the number of args, but this itself would not be a signal to avoid further processing, but errors could be used.

Other approaches

Several of these approaches may use subsystems to define the parsing and carry data between this custom aspect of parsing and execution.

Have the parser support directives

This may be difficult in API review because it is specific to an unusual feature.

Allow custom token handling

This is similar or the same as the previous custom parsing that we would like to avoid. Would it be easier to manage if it was during Tokenize? This would be similar to putting the code currently commented out in Tokenize into custom Func<>s

?

[KalleOlaviNiemitalo]

They also solve an interesting problem: how can a CLI parser add default behavior without breaking existing CLI designs?

IIRC, the GNU project attempted something similar: made the Bash shell add an environment variable declaring which command-line arguments were expanded from globs and should not be treated as options, and made getopt_long check this environment variable. The name of the environment variable included the process ID, to prevent it from affecting command-line parsing in grandchild processes. But they disabled the whole feature later.

(Such an environment variable would be much more difficult to implement on Windows, because CreateProcess doesn't let you run application code between fork and exec.)

[KathleenDollard]

I am editing the proposal to change from altering the input string to accepting an array of args and supplying an integer indicating how many args should be skipped.

This combines the simplicity of reusing the subsystem design with a simple way to indicate to the parser what has been handled.

I'm also that preprocessing in a subsystem is just Initialize.

Response files and environment variables, other than setting new environment variables as in the existing directives, may be separated from this proposal.