Confusion around managing flags like `--version`

sbarrios93 commented 7 months ago

I've been impressed with Tyro's capabilities so far. However, I've encountered some ergonomic challenges that I hope to get some help with.

Related Issue: This issue seems to be related to issue #89. Although I attempted the solutions suggested there, I was unable to resolve my problem.

The Problem: I'm in the process of creating a CLI with two subcommands: init and run. Each subcommand requires its own set of arguments. Additionally, I want the CLI to accept global flags such as --version or --status. Here's my initial setup:

Initial Setup

from typing import Annotated, Optional, Union

import tyro
from pydantic import BaseModel

class Init(BaseModel):
    env: str = ".env"
    force: bool = False

class Run(BaseModel):
    headless: bool
    days_back: int
    short_items: bool
    words_per_item: int

class Args(BaseModel):
    subcommand: Union[Init, Run]
    version: bool = False

def entrypoint() -> None:
    tyro.extras.set_accent_color("bright_yellow")
    args = tyro.cli(Annotated[Args, tyro.conf.OmitSubcommandPrefixes])
    print(args)

(.venv) python cli.py --help     
usage: cli.py [-h] [--version | --no-version] {init,run}

╭─ options ─────────────────────────────────────────╮
│ -h, --help        show this help message and exit │
│ --version, --no-version                           │
│                   (default: False)                │
╰───────────────────────────────────────────────────╯
╭─ subcommands ─────────────────────────────────────╮
│ {init,run}                                        │
│     init                                          │
│     run                                           │
╰───────────────────────────────────────────────────╯

Running cli.py --help displays the help text as expected. However, I've noticed that the --no-version flag is automatically generated and doesn't align with my requirements. I am looking for a way to disable this automatic generation.

Moreover, when I attempt to use the --version flag independently, it doesn't function as anticipated:

(.venv) python cli.py --version
╭─ Required options ───────────────────────────────╮
│ The following arguments are required: {init,run} │
│ ──────────────────────────────────────────────── │
│ For full helptext, run cli.py --help             │
╰──────────────────────────────────────────────────╯

This command incorrectly demands a subcommand (init or run), which should not be the case for a version check.

When running --version under one of the subcommands, an error is thrown out, which makes sense but the error doesn't.

(.venv) python cli.py init --version
╭─ Unrecognized options ───────────────────────╮
│ Unrecognized or misplaced options: --version │
│ ──────────────────────────────────────────── │
│ Perhaps you meant:                           │
│     --version, --no-version                  │
│         (default: False)                     │
│             in cli.py --help                 │
│ ──────────────────────────────────────────── │
│ For full helptext, run cli.py --help         │
╰──────────────────────────────────────────────╯

Optional Subcommands

To work around this, I tried making the subcommand optional:

import tyro
from pydantic import BaseModel

class Init(BaseModel):
    env: str = ".env"
    force: bool = False

class Run(BaseModel):
    headless: bool
    days_back: int
    short_items: bool
    words_per_item: int

class Args(BaseModel):
    subcommand: Optional[Union[Init, Run]] = None
    version: bool = False

def entrypoint() -> None:
    tyro.extras.set_accent_color("bright_yellow")
    args = tyro.cli(Annotated[Args, tyro.conf.OmitSubcommandPrefixes])
    print(args)

if __name__ == "__main__":
    entrypoint()

Applying an Optional type with default value = None allows --version to be run by itself but also appends a subcommand None option, which doesn't make sense in this case.

(.venv) python cli.py --version
subcommand=None version=True

(.venv) python cli.py --help
usage: cli.py [-h] [--version | --no-version] [{init,run,None}]

╭─ options ───────────────────────────────────────────────╮
│ -h, --help              show this help message and exit │
│ --version, --no-version                                 │
│                         (default: False)                │
╰─────────────────────────────────────────────────────────╯
╭─ optional subcommands ──────────────────────────────────╮
│ (default: None)                                         │
│ ─────────────────                                       │
│ [{init,run,None}]                                       │
│     init                                                │
│     run                                                 │
│     None                                                │
╰─────────────────────────────────────────────────────────╯

Regarding #89, I've tried modifying the alternatives stated here and here with no success.

1st try

import dataclasses
from typing import Tuple, Union

import tyro

@dataclasses.dataclass
class Checkout:
    """Check out a branch."""

    x: int

@dataclasses.dataclass
class Commit:
    """Commit something."""

    y: int

@dataclasses.dataclass
class Args:
    version: bool = False
    """Print version and exit."""

global_args, checkout_or_commit = tyro.cli(
    Tuple[
        Args,
        Union[Checkout, Commit],
    ]
)
print(global_args, checkout_or_commit)

(.venv) python cli.py  --0.version
╭─ Required options ──────────────────────────────────────────╮
│ The following arguments are required: {1:checkout,1:commit} │
│ ─────────────────────────────────────────────────────────── │
│ For full helptext, run cli.py --help                        │
╰─────────────────────────────────────────────────────────────╯

2nd try

import dataclasses
from typing import Annotated, Tuple, Union

import tyro

@dataclasses.dataclass
class Checkout:
    """Check out a branch."""

    x: int

@dataclasses.dataclass
class Commit:
    """Commit something."""

    y: int

@dataclasses.dataclass
class Args:
    version: bool = False
    """Print version and exit."""

global_args, checkout_or_commit = tyro.cli(
    Annotated[
        Tuple[
            Union[
                Checkout,
                Commit,
            ],
            Args,
        ],
        tyro.conf.OmitArgPrefixes,
        tyro.conf.OmitSubcommandPrefixes,
    ]
)
print(global_args, checkout_or_commit)

(.venv) python cli.py  --version
╭─ Required options ──────────────────────────────────────╮
│ The following arguments are required: {checkout,commit} │
│ ─────────────────────────────────────────────────────── │
│ For full helptext, run cli.py --help                    │
╰─────────────────────────────────────────────────────────╯

I'm confused here. Not sure if I'm misunderstanding something from the documentation or how to apply different options, but seems that applying flags that i) do not create --no-x versions and ii) can be applied alone outside any command, should be fairly straightforward on a CLI tool.

Thanks!

brentyi commented 7 months ago

Hi, thanks for filing this issue! I'm working on a deadline right now and will be mostly offline until ~Friday (I can follow up after if you have questions), but here are some initial thoughts. In general I think tyro's working as designed, it is though a fairly opinionated tool so it's possible it's not the best fit if you need more control.

(1) For the [--flag | --no-flag] creation: unfortunately, this is not configurable right now. I think the current behavior is a sensible default, the motivation is outlined in https://github.com/brentyi/tyro/issues/48#issuecomment-1535202965. We could consider making this configurable, but I'd need to think about it since this is primarily an aesthetic change.

(2)

class Args(BaseModel):
    subcommand: Union[Init, Run]
    version: bool = False
[tyro.cli(Args)] incorrectly demands a subcommand (init or run), which should not be the case for a version check.

Unfortunately, I don't think there's an alternative behavior that makes sense. When passed a type like Args, the CLI tyro generates is intended to match the semantics of the Python constructor to the closest extent possible.

In this case, the Python constructor has specified:

that there's a required argument subcommand, which has the type Init | Run.
that there's an optional argument version. You can choose to pass this in or not.

To me this seems reflected 1:1 in the behavior of the CLI you've generated. We just need to put a value in for subcommand to construct an Args object.

For how to achieve the behavior you want: I can think about it but specifying that there's a set of subcommands that are optional only when --version is passed in may be beyond what tyro is capable of expressing.

(3)

When running --version under one of the subcommands, an error is thrown out, which makes sense but the error doesn't.

(.venv) python cli.py init --version
╭─ Unrecognized options ───────────────────────╮
│ Unrecognized or misplaced options: --version │
│ ──────────────────────────────────────────── │
│ Perhaps you meant:                           │
│     --version, --no-version                  │
│         (default: False)                     │
│             in cli.py --help                 │
│ ──────────────────────────────────────────── │
│ For full helptext, run cli.py --help         │
╰──────────────────────────────────────────────╯

The error message is saying that the argument is not recognized in its current location, and in particular it might be misplaced. The --version flag exists in the cli.py, but not the cli.py init (sup)parser that you've pass it to.

Open to suggestions if you have ideas on how to clarify!

(4)

Applying an Optional type with default value = None allows --version to be run by itself but also appends a subcommand None option, which doesn't make sense in this case.

This is happening because you've annotated subcommand as Init | Run | None. As a result, tyro provides three options: init, run, and None.

sbarrios93 commented 7 months ago

Hey, thanks for the detailed response and for going through my message.

I understand there are some limitations and I'd be more than happy to help with PR's if you want to land on a desired behavior for Tyro around this.

For how to achieve the behavior you want: I can think about it but specifying that there's a set of subcommands that are optional only when --version is passed in may be beyond what tyro is capable of expressing.

Ultimately, I want to pass flags that behave like --help: provide some info and exit. That's it.

Let me know if you have an idea of how can this be achieved! Thanks!

brentyi commented 7 months ago

Thanks for the offer! I can think about it, but I'm not sure this is well-supported even in more expressive libraries like argparse.

If you want a hacky solution one option is to check sys.argv explicitly for --verbose before tyro.cli() is called.

stevapple commented 3 months ago

If you want a hacky solution one option is to check sys.argv explicitly for --verbose before tyro.cli() is called.

How to add the help text in this case though...?

brentyi / tyro

Confusion around managing flags like `--version` #132

Initial Setup

Optional Subcommands