[QUESTION] How to wrap commands

[mmcenti]

First check

• [x] I used the GitHub search to find a similar issue and didn't find it.
• [x] I searched the Typer documentation, with the integrated search.
• [x] I already searched in Google "How to X in Typer" and didn't find any information.
• [x] I already searched in Google "How to X in Click" and didn't find any information.

Description

How can I wrap a command and extend the options?

I am able to do this via Click but haven't figured out a way to make it work. I want to be able to extend certain commands to give extra options when they are annotated.

Lets say I have a command defined as:

@app.command()
def hi(name: str = typer.Option(..., '--name', '-n', prompt=True, help="Name of person to say hi to"),):
    """ Say hello. """

    print(f"Hello, {name}!")

I can call this via python cli.py hi. Want I want is the ability to wrap the hi function to include a city and state if I wanted. Example:

def from_city(func,) -> "wrapper":
    @wraps(func)
    def wrapper(
        city: str = typer.Option(..., '--city', '-c', prompt=True, help='The name of the city to say hi from'),
        state: str = typer.Option("VA", '--state', '-s', help="The state you are saying hi from"),
    ):
        """ Setup for finding city. """
        # <do some stuff>
        return None
    return wrapper

@app.command()
@from_city
def hi_city(name: str = typer.Option(..., '--name', '-n', prompt=True, help="Name of person to say hi to"),):
    """ Say hello. """

    print(f"Hello, {name}. Welcome from {kwargs['city']}, {kwargs['state']}!")

By adding the @from_city annotation, I want the additional city/state options to be available.

Additional context

I can do this in Click with the following code:

def from_city(func) -> "wrapper":
    @click.pass_context
    @click.option(
        "--city",
        "-c",
        show_default=True,
        help="The name of the city to say hi from",
    )
    @click.option(
        "--state",
        "-s",
        default="VA",
        show_default=True,
        help="The state you are saying hi from",
    )
    @__wraps(func)
    def wrapper(*args, **kwargs):
        # Grab the context and put it in kwargs so the wrapped function has access
        kwargs["ctx"] = args[0]

        return

    return wrapper

@cli.command()
@from_city
@click.option(
    "--name",
    "-n",
    default="~/Downloads/pyshthreading.yaml",
    help="Location on disk to the cf template",
)
def test(*args, **kwargs):
    print(f"Hello, {kwargs['name']}. Welcome from {kwargs['city']}, {kwargs['state']}!")

Is there a way to do this in Typer?

[sathoune]

I have a solution for you.

The main problem here is the way Typer knows about command parameters. It uses function signature and type hints. hi_city has one parameter of name and that's what it expects. And you use functools.wraps so the parameters stay the same. To add more you have to modify the function signature to include the parameters you want to reuse in from_city.

I was tinkering how to do that elegantly and I found an article which does exactly that: https://chriswarrick.com/blog/2018/09/20/python-hackery-merging-signatures-of-two-python-functions/

The author created package merg-args: you can install it, you can check out the bottom of the article.

The secondary problem is how to pass the new parameters. You cannot use kwargs, because Typer does not support this. You can use typer.Context (click.Context in disguise) to access them.

I used the mention package and the Context to achieve your goal:

from typing import Callable

import typer
from merge_args import merge_args

app = typer.Typer()

def from_city(
    func: Callable,
) -> "wrapper":
    @merge_args(func)
    def wrapper(
        ctx: typer.Context,
        city: str = typer.Option(
            ..., "--city", "-c", prompt=True, help="The name of the city to say hi from"
        ),
        state: str = typer.Option(
            "VA", "--state", "-s", help="The state you are saying hi from"
        ),
        **kwargs,
    ):
        """Setup for finding city."""
        return func(
            ctx=ctx,
            **kwargs
        )

    return wrapper

@app.command()
@from_city
def hi_city(
    ctx: typer.Context,
    name: str = typer.Option(
        ..., "--name", "-n", prompt=True, help="Name of person to say hi to"
    ),
):
    """Say hello."""
    kwargs = ctx.params
    print(f"Hello, {name}. Welcome from {kwargs['city']}, {kwargs['state']}!")

if __name__ == "__main__":
    app()

ctx.params stores all the parameters that function was invoked with, so name too and you can use it, as you wanted to use **kwargs.

[sathoune]

You can actually also use the Typer callback. It will be less obvious what arguments there are but looks like less code and does not require magic like the above solution.

import typer

app = typer.Typer()

@app.callback()
def extras(
    ctx: typer.Context,
    city: str = typer.Option(
        ..., "--city", "-c", prompt=True, help="The name of the city to say hi from"
    ),
    state: str = typer.Option(
        "VA", "--state", "-s", help="The state you are saying hi from"
    ),
):
    ctx.obj = ctx.params
    pass

@app.command()
def hi_city(
    ctx: typer.Context,
    name: str = typer.Option(
        ..., "--name", "-n", prompt=True, help="Name of person to say hi to"
    ),
):
    """Say hello."""
    kwargs = ctx.obj
    print(f"Hello, {name}. Welcome from {kwargs['city']}, {kwargs['state']}!")

if __name__ == "__main__":
    app()

[mmcenti]

@captainCapitalism thank you for this! I had never heard of merge_args. I believe this will work for our use case. Will close after doing some more testing.

[lovetoburnswhen]

You can actually also use the Typer callback. It will be less obvious what arguments there are but looks like less code and does not require magic like the above solution.

import typer

app = typer.Typer()

@app.callback()
def extras(
    ctx: typer.Context,
    city: str = typer.Option(
        ..., "--city", "-c", prompt=True, help="The name of the city to say hi from"
    ),
    state: str = typer.Option(
        "VA", "--state", "-s", help="The state you are saying hi from"
    ),
):
    ctx.obj = ctx.params
    pass

@app.command()
def hi_city(
    ctx: typer.Context,
    name: str = typer.Option(
        ..., "--name", "-n", prompt=True, help="Name of person to say hi to"
    ),
):
    """Say hello."""
    kwargs = ctx.obj
    print(f"Hello, {name}. Welcome from {kwargs['city']}, {kwargs['state']}!")

if __name__ == "__main__":
    app()

This one wouldn't work for apps that already have a callback

[robinbowes]

I found a was getting a error from pyflakes with @captainCapitalism 's example code. Removing -> "wrapper" seems to fix the error.

I also found that it didn't work with sub-commands or multiple commands - the commands were not listed correctly. functools.wraps fixes this.

Here's a revised example with two commands:

from typing import Callable

import typer
from functools import wraps
from merge_args import merge_args

app = typer.Typer()

def from_city(
    func: Callable,
) -> "wrapper":  # noqa: F821
    @merge_args(func)
    @wraps(func)
    def wrapper(
        ctx: typer.Context,
        city: str = typer.Option(
            ...,
            "--city",
            "-c",
            prompt=True,
            help="The name of the city to say hi from"
        ),
        state: str = typer.Option(
            "VA", "--state", "-s", help="The state you are saying hi from"
        ),
        **kwargs,
    ):
        """Setup for finding city."""
        return func(ctx=ctx, **kwargs)

    return wrapper

@app.command()
@from_city
def hi_city(
    ctx: typer.Context,
    name: str = typer.Option(
        ..., "--name", "-n", prompt=True, help="Name of person to say hi to"
    ),
):
    """Say hello."""
    kwargs = ctx.params
    print(f"Hello, {name}. Welcome from {kwargs['city']}, {kwargs['state']}!")

@app.command()
@from_city
def bye_city(
    ctx: typer.Context,
    name: str = typer.Option(
        ..., "--name", "-n", prompt=True, help="Name of person to say bye to"
    ),
):
    """Say goodbye."""
    kwargs = ctx.params
    print(
        f"Goodbye, {name}. Welcome from {kwargs['city']}, {kwargs['state']}!"
    )

if __name__ == "__main__":
    app()

I think this is something I can work with.

[robinbowes]

Ugh, found a problem. I tried adding this to allow the common options to be specified before the command:

@app.callback()
@from_city
def main(
    ctx: typer.Context,
) -> None:
    # ctx.obj = Common(profile=profile, region=region)
    pass

And, while it does allow the from_city options to be specified, they are not recognised by the options in the sub-commands.

For example, this still prompts for city:

$ python3 main.py --city foo hi-city --name bar

[Zaubeerer]

As mentioned in https://github.com/tiangolo/typer/issues/405#issuecomment-1191719026 I would also be interested in an elegant solution to this. :)

[jimkring]

Here's an addition to @robinbowes code that uses a pydantic model (1) to make it easy to declare (2) and obtain the options (3) in a way that's a little bit more statically typed (4). Basically, it avoids having to access the options from a dictionary by name/string. This still feels like too many hoops to jump through, but it feels slightly easier to maintain. Also, a vanilla dataclass (rather than a pydantic model) might work too, but I didn't go that route.

from typing import Callable

from pydantic import BaseModel
import typer
from merge_args import merge_args

app = typer.Typer()

# 1) define the common options in a pydantic model
class FromCity(BaseModel):
    city: str = typer.Option(
        ..., "--city", "-c", prompt=True, help="The name of the city to say hi from"
    )
    state: str = typer.Option(
        "VA", "--state", "-s", help="The state you are saying hi from"
    )
    class Config:
        arbitrary_types_allowed = True

def from_city(
    func: Callable,
) -> "wrapper":
    @merge_args(func)
    def wrapper(
        ctx: typer.Context,
        # 2) get the TyperOptions for each of the options
        city: str = FromCity().city,  #  <-- here
        state: str = FromCity().state,  #  <-- here
        **kwargs,
    ):
        return func(ctx=ctx, **kwargs)
    return wrapper

@app.command()
@from_city
def hi_city(
    ctx: typer.Context,
    name: str = typer.Option(
        ..., "--name", "-n", prompt=True, help="Name of person to say hi to"
    ),
):
    """Say hello."""
    # 3) Convert the arguments into an object / instance of the pydantic model
    from_city = FromCity(**ctx.params)
    # 4) Access them as attributes of the objects
    print(f"Hello, {name}. Welcome from {from_city.city}, {from_city.state}!")

if __name__ == "__main__":
    app()

[mzebrak]

@jimkring Which version of python are u using?

I get into some errors with your solution on 3.10 like NameError: name 'typer' is not defined. But when I comment both of the ctx: typer.Context, lines, It seems working (but of course partially since there is no access to the Context obj)

Seems like this one is really related to the changed behaviour of typing in 3.10. Also when i use somewhere tyhe Optional annotation i get: NameError: name 'Optional' is not defined and it comes from the internals of .pyenv/versions/3.10.9/lib/python3.10/typing.py:694 in _evaluate

====== EDIT: After removing from __future__ import annotations from my code, it works as it should \ (also @wraps(func, assigned=["__module__", "__name__", "__doc__", "__anotations__"]) decorator from functools is needed). But that means there is a problem which will be visible in version 3.11 when annotations will be included in the standard lib.

So to summarize - IDK if this is an issue with the Typer or with the merge_args, but when it comes to eval() call, deep inside typing.py, it throws an error when from __future__ import annotations is included

[NikosAlexandris]

Here's an addition to @robinbowes code that uses a pydantic model (1) to make it easy to declare (2) and obtain the options (3) in a way that's a little bit more statically typed (4). Basically, it avoids having to access the options from a dictionary by name/string. This still feels like too many hoops to jump through, but it feels slightly easier to maintain. Also, a vanilla dataclass (rather than a pydantic model) might work too, but I didn't go that route.

from typing import Callable

from pydantic import BaseModel
import typer
from merge_args import merge_args

app = typer.Typer()

# 1) define the common options in a pydantic model
class FromCity(BaseModel):
    city: str = typer.Option(
        ..., "--city", "-c", prompt=True, help="The name of the city to say hi from"
    )
    state: str = typer.Option(
        "VA", "--state", "-s", help="The state you are saying hi from"
    )
    class Config:
        arbitrary_types_allowed = True

def from_city(
    func: Callable,
) -> "wrapper":
    @merge_args(func)
    def wrapper(
        ctx: typer.Context,
        # 2) get the TyperOptions for each of the options
        city: str = FromCity().city,  #  <-- here
        state: str = FromCity().state,  #  <-- here
        **kwargs,
    ):
        return func(ctx=ctx, **kwargs)
    return wrapper

@app.command()
@from_city
def hi_city(
    ctx: typer.Context,
    name: str = typer.Option(
        ..., "--name", "-n", prompt=True, help="Name of person to say hi to"
    ),
):
    """Say hello."""
    # 3) Convert the arguments into an object / instance of the pydantic model
    from_city = FromCity(**ctx.params)
    # 4) Access them as attributes of the objects
    print(f"Hello, {name}. Welcome from {from_city.city}, {from_city.state}!")

if __name__ == "__main__":
    app()

How does this perform ? Is it slowing down building the CLI, in comparison to normal functions with normal signatures ?