argparse: support mutually exclusive groups like ( -a | (-b | -c) )

[calestyo]

Feature or enhancement

Right now, when adding arguments via add_mutually_exclusive_group(), every one of them is mutually exclusive to each other. It would be nice if one could add a group like in ( -a | (-b | -c) ), so that one can either use -a (alone) or -b and/or -c, but not -a together with -b and/or -c.

Pitch

I'd say that the above is a rather common use case and that's the sole justification for having it ;-)

Of course I realise that one could argue like this for many possible dependency/conflict scenario amongst the arguments and the question is always what argparse should support out of the box, and what is overkill for it.

And if one would combine this with what's asked for in #55797, things would get even more complex. E.g. -a could be mutually exclusive with (-b | -c) and -b could depend on -x.

Previous discussion

Not sure whether any discussion has already taken place. I've looked through all open/closed github issues and at least by their titles I found none which seems exactly the same.

[hpaulj]

Looks like this is a subset of what I tried to implement for https://github.com/python/cpython/issues/55797. There I was aiming for nested groups that can test all possible logic combinations. Nesting groups within groups was easy enough, and implementing a test using the seen_actions set/list wasn't too hard. But adding it to the usage format required a major rewrite of that method. As it stands usage formatting too brittle.

Since then various issues have led an increased restriction on nesting groups (action and mutual).

I suppose an alternative approach would be to alter the mutually_exclusive_group definition and test to handle groups of Actions (not 'action_groups'). But that I don't think that solves the usage formatting issue.

It's been a long time since I worked on that issue, but as you can see from my earlier posts I gave up because it was too complex.

One final thought - I have suggested refactoring the tests at the end _parse_known_args to make it easier to give the user access to the seen_actions list, and do their own logic tests. For now users have to use if args.foobar is None kinds of tests to check whether something had been 'seen' or not.

[calestyo]

Well,... for the really complex cases,... is it even necessary/reasonable to have a usage print out? I mean that would get just awfully unreadable.

I think it would already be beneficial if such dependencies would be supported simply in terms of parsing. Most programs that I know, which have such complex usage modes, simply don't list them anymore in their manpage (I guess mostly because they'd be barely readable anyway).

[Eutropios]

I agree that this feature would be incredibly useful for the implementation of command line interfaces. It should definitely be included, especially since this module is in the standard library.

[camball]

While I understand this has been deprecated since 3.11 (in #30098), I really feel this is a big oversight as there's many cases where this is useful. Seems to me like a fundamental part of properly parsing arguments.

For example, there's no way for me to handle the case where I want my users to be able to upload either a key/value pair, or a JSON file, but if a key/value pair is specified, the value can either be read as a string, or a "from-file" argument can be passed to read from a file.

my_cmd (--from-json <file> | key (value | --from-file <file>))

My point here is that I feel like the current approach of just deprecating the feature because it was unintentionally inherited from a parent class is just a band-aid for the actual problem of incomplete parsing functionality.

[hpaulj]

Nesting a group within a mutually exclusive group had two main problems.

1) people were trying to put argument groups within the MXG, thinking it would give them some sort of and/or logic. Argument_groups only affect the help formatting, and do nothing to the parsing.

2) people complained that usage formatting was messed up when they put a MXG within a MXG. That the usage formatter is brittle is well known. We've patched it in various ways, but it really needs a major rewrite.

It also turned out that nesting MXGs doesn't do anything special - A xor (B xor C) is just (A xor B xor C). During parsing it's just one big xor group.

I don't understand your example:

 (--from-json <file> | key (value | --from-file <file>))

What's this key? '--from-json' is a flag, an optional's key. Argparse does not handle generic 'key value' entries. Without the dash, key looks just like a positional argument string. It is possible to accept pairs of strings like key value' orkey=value' as plain strings (possibly with nargs=2, and split them up after parsing.

A MXG can have a ? positional argument (last of the group):

(--from-json <file> | --from-file <file> | value)

would be a 3 way xor group, accepting either of the --from flags, or a plain positional value.

While users can imagine all kinds of parsing mixes, argparse has to consider input (clear, unambiguous - where possible), usage and help display, error display, and the parsing itself. I often ask developers - what's the usage you want to show? Will it be understandable to your end users, or even yourself six months from now? The primary goal of argparse is to clearly parse what your end user wants. It doesn't have to flag all 'wrong' mixes of inputs; it's ok to do some post-parsing checkup and cleanup.

paulj3

On Sun, Nov 5, 2023 at 12:45 AM Cameron Ball @.***> wrote:

While I understand this has been deprecated since 3.11 (in #30098 https://github.com/python/cpython/pull/30098), I really feel this is a big oversight as there's many cases where this is useful. Seems to me like a fundamental part of properly parsing arguments.

For example, there's no way for me to handle the case where I want my users to be able to upload either a key/value pair, or a JSON file, but if a key/value pair is specified, the value can either be read as a string, or a "from-file" argument can be passed to read from a file.

my_cmd (--from-json | key (value | --from-file ))

My point here is that I feel like the current approach of just deprecating the feature because it was unintentionally inherited from a parent class is just a band-aid for the actual problem of incomplete parsing functionality.

— Reply to this email directly, view it on GitHub https://github.com/python/cpython/issues/101337#issuecomment-1793663969, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAITB6GGAJKU3U7DLFQ7V2LYC472RAVCNFSM6AAAAAAUHC7HHWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOJTGY3DGOJWHE . You are receiving this because you commented.Message ID: @.***>

[stmlange]

Hello Python World, bumped into a similar question problem where it would be nice to supported more complex nested groups like: ( -a | (-b & -c) | (-d & -e) ) in short one can either specify (-a) XOR (-b and -c) XOR (-d and -e).

Calling add_argument_group() or add_mutually_exclusive_group() on a mutually exclusive group is deprecated since 3.11 so I don't see how one could write such complex nested group.

Thanks :-)

[hpaulj]

Even without the deprecation, nesting an argument_group within a mutually_exclusive_group does not create an and group within the xor. It just adds the arguments to a flat xor.

Your usage will just be:

 ( -a | -b | -c | -d | -e )

The 3.11 deprecation is just a way of telling programmers that their setup nesting is not doing what they want it to; it doesn't actually remove any features.

As noted in my previous comments, there's more to this issue than nesting the groups in the setup. argument_groups are not and counterparts to the xor. The argument_group defines a help formatting group. For parsing purposes it just adds its arguments to whatever parser or group it was 'nested' in.

I explored this issue many years ago in an issue that I referenced above. I may have been overly ambitious back then, but as far as I know no one has tried to write a simpler patch.

Correction

When I add an action group (with g and k) to a MXG, the usage I get is just flat

  [-h] [-f F] [-g G] [-k K]

and the action group is not even visible in the help lines. It's even more of deadend feature than I realized. That the nesting was even possible during setup is just an accident of how methods are inherited by _ActionContainer subclasses, and was never intended as a functional feature. The similarity in names, argument_group and mutually_exclusive_group misleads users who wish for more than what's documented.