argparse group nesting lost on inheritance

[e1d29b37-30c5-4023-830b-f8e2f5354e80]

BPO	16807
Nosy	@dougalsutherland, @iritkatriel
Files	argparse_mutex_parent.patch: patch fixing the problem and adding a test

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = None closed_at = None created_at = labels = ['type-bug', 'library', '3.9', '3.10', '3.11'] title = 'argparse group nesting lost on inheritance' updated_at = user = 'https://github.com/dougalsutherland' ``` bugs.python.org fields: ```python activity = actor = 'iritkatriel' assignee = 'none' closed = False closed_date = None closer = None components = ['Library (Lib)'] creation = creator = 'dougalsutherland' dependencies = [] files = ['28473'] hgrepos = [] issue_num = 16807 keywords = ['patch'] message_count = 5.0 messages = ['178459', '222934', '223047', '223158', '408222'] nosy_count = 4.0 nosy_names = ['bethard', 'paul.j3', 'dougalsutherland', 'iritkatriel'] pr_nums = [] priority = 'normal' resolution = None stage = None status = 'open' superseder = None type = 'behavior' url = 'https://bugs.python.org/issue16807' versions = ['Python 3.9', 'Python 3.10', 'Python 3.11'] ```

Linked PRs

• gh-125210
• gh-125308
• gh-125309

[e1d29b37-30c5-4023-830b-f8e2f5354e80]

If you wrap a mutually exclusive group inside an argument group in an argparse.ArgumentParser, and then use parents= to inherit from that parser, the non-exclusive argument group is forgotten in the help output, and the arguments move to the default "optional arguments" section. For example:

    # construct the parser with a mutually exclusive group
    >>> import argparse
    >>> parent = argparse.ArgumentParser(add_help=False)
    >>> group = parent.add_argument_group('the group')
    >>> group.add_argument('--foo') and None
    >>> mutex = group.add_mutually_exclusive_group()
    >>> mutex.add_argument('-a', action='store_true') and None
    >>> mutex.add_argument('-b', action='store_true') and None
    >>> parent.print_help()
    usage: [--foo FOO] [-a | -b]

    the group:
      --foo FOO
      -a
      -b

    # now try to inherit from it; "the group" is forgotten for
    # mutex arguments, but remains for others
    >>> argparse.ArgumentParser(add_help=False, parents=[parent]).print_help()
    usage: [--foo FOO] [-a | -b]

    optional arguments:
      -a
      -b

    the group:
      --foo FOO

I see the same behavior on 2.7.3 and 3.3.0.

The problem is that argparse._ActionsContainer._add_container_actions always adds mutex groups to the top level, rather than to the equivalent of their _container attribute. The attached patch fixes this, and adds a test based on the formatted output (almost identical to the test_groups_parents test).

One thing about the patch: it assumes that the _container attribute of all the mutex groups will be either the container argument to _add_container_actions or an argument group that has been processed in group_map. If this is not the case, it'll fail with either an AttributeError or a KeyError. I don't know when this would happen, or if it's common enough that it's worth checking for more explicitly.

[83d2e70e-e599-4a04-b820-3814bbdb9bef]

@Dougal sorry about the delay in getting back to you.

[7a064fe6-c535-4d80-a11f-a04ed39056c5]

To put this issue in perspective:

• There's nothing in the documentation about nesting a mutually exclusive group in an argument group.

• There are prominent notes in the documentation about MEGs not taking title and description. The '_add_container_actions' code has a warning that something needs to change if they do acquire such attributes.

• However there is a class in test_argparse.py that does nest an MEG in an argument group, which in effect gives it a title. TestMutuallyExclusiveInGroup That's the pattern that Dougal is using.

• http://bugs.python.org/issue17218 support title and description in argparse add_mutually_exclusive_group

proposes adding these attributes to MEG. There I proposed doing so via this nested group mechanism. So that patch requires this one to work correctly with parents.

• http://bugs.python.org/issue11588 I am exploring the implementation of UsageGroups, a generalization of MEG that allow other group tests, and nesting. There too title and description come via nesting in an ArgumentGroup.

• Groups and parents are confusing to beginning users. In StackOverflow questions 'argparse' users often confuse argument groups and mutually exclusive groups, expecting to be able to nest one inside the other. Currently that nesting only works one way, and for the limited purpose illustrated here.

[7a064fe6-c535-4d80-a11f-a04ed39056c5]

The subcommands grouping mechanism proposed in http://bugs.python.org/issue9341 probably does not work with [parents] either.

This _add_container_actions method is brittle. It has to know too much about the structure of a parser and its groups. Any change to that structure could break this [parents] mechanism.

In a refactoring, each 'add_xxx' method would have a 'copy_xxx' companion method.

[iritkatriel]

Reproduced on 3.11:

>>> argparse.ArgumentParser(add_help=False, parents=[parent]).print_help()
usage: [--foo FOO] [-a | -b]

options:
  -a
  -b

the group:
  --foo FOO

[serhiy-storchaka]

@djsutherland, is it your patch? Are you interesting in creating a PR?

[serhiy-storchaka]

The page of the original author of the patch @dougalsutherland was moved to @djsutherland, but that page has different name, and I do not know if they are the same person or relatives. So I used the old name to credit the author in #125210.

[djsutherland]

Oh wow, this has been a while!

I am the same person; I changed my name years ago. Please change the ACKS in the other patch to Danica J Sutherland; thanks @serhiy-storchaka!

[serhiy-storchaka]

Done. Thank you for your contribution, Danica, and sorry it took too long to review your patch.