Fetching sys.stdin using annotations in a command

sbordeynemics commented 2 years ago

First Check

[X] I added a very descriptive title to this issue.
[X] I used the GitHub search to find a similar issue and didn't find it.
[X] I searched the Typer documentation, with the integrated search.
[X] I already searched in Google "How to X in Typer" and didn't find any information.
[X] I already read and followed all the tutorial in the docs and didn't find an answer.
[X] I already checked if it is not related to Typer but to Click.

Commit to Help

[X] I commit to help with one of those options 👆

Example Code

# This does not work as far as I know in the current version of typer
import typer

app = typer.Typer()

@app.command()
def cat(data: typer.StdIn):
    '''Prints all the data piped into it from stdin'''
    for line in data:
        typer.echo(line)

if __name__ == '__main__':
    app()

Description

I would like to have typer integrate seemlessly with sys.stdin. Currently, it is quite complicated to integrate sys.stdin in a typer app. You can obviously use something like :

import sys
import typer

app = typer.Typer()

@app.callback(invoke_without_command=True)
def main(value: typer.FileText = typer.Argument(..., allow_dash=True)):
    """Echo out the input from value."""
    value = sys.stdin.read() if value == "-" else value.read()
    typer.echo(value)

if __name__ == "__main__":
    app()

See this issue : #156

But it is definitely not seemless.

Wanted Solution

The goal would be to have something like in the code example provided, direct access to stdin from the argument type (which is handy when designing CLIs so that they can be used in conjunction with other unix tools -- like cut, cat, sed or find for instance)

Wanted Code

# This does not work as far as I know in the current version of typer
import typer

app = typer.Typer()

@app.command()
def cat(data: typer.StdIn):
    '''Prints all the data piped into it from stdin'''
    for line in data:
        typer.echo(line)

if __name__ == '__main__':
    app()

Alternatives

Use the aforementionned workaround, using '-' as the marker for an stdin input.

I also tried reading from sys.stdin directly, but it doesn't integrate well in a typer app (especially around the coding style)

Operating System

Linux, Windows, macOS

Operating System Details

Windows : Windows 10 Professional (latest build)
Mac OS : Monterey 12.0.1
Linux : Ubuntu 20.04 LTS, Fedora 34

Typer Version

0.3.2

Python Version

3.9.9

Additional Context

No response

jd-solanki commented 2 years ago

I also want this for my automation scripts

jd-solanki commented 2 years ago

Hi,

I found a solution to the problem I was working on. My automation script takes and string as CLI argument and writes it after couple of seconds.

python write_text.py "write"

# I converted it in the global script so I can run it from anywhere. Now, I use it like:
write_text "write"

However, there were cases where string I wanted to auto write was coming from some other bash command output so I wanted my script to run in both cases:

write_text "write"

# and, below example read from stdin

echo "write" | write_text

# Error if you don't pass CLI argument or don't provide stdin

This was challenging but thanks to this comment, I got an idea and here's example that let you pass argument and read from stdin.

import sys
import typer

def main(
    name: str = typer.Argument(
        ... if sys.stdin.isatty() else sys.stdin.read().strip()
    ),
):
    typer.echo(f"Hello {name}")

if __name__ == "__main__":
    typer.run(main)

I hope this will help you in resolving the issue

theelderbeever commented 1 year ago

Not sure why but the above answer doesn't seem to work for me unfortunately...

@app.command()
def echo(
    msg: str = typer.Argument(... if sys.stdin.isatty() else sys.stdin.read()),
):
    print(msg)

The following test always prints a blank line. Same occurs when cat-ing a file.

echo hello | myapp echo

So I would definitely support an option that can read - from stdin and any suggestions from anyone else.

jd-solanki commented 1 year ago

@theelderbeever above code example is full minimal app/script.

Have you tried running that app?

blakeNaccarato commented 1 year ago

A slightly different approach/workaround looks like this. You can pass an array of strings, or, say an array of filenames into the Python script if you dot-source common.ps1 below and hello.py is on your PYTHONPATH or is a module. You could make the -Module parameter more flexible, and expand the example further, but that's the gist of it.

Of course there's no real parallelism here, the Python interpreter fires up and runs with every input piped to it, there's no benefit of setup/teardown e.g. begin{} and end{} clauses in PowerShell's pipeline orchestration mechanisms.

The upside is you don't need to do anything special to your Python module, since all logic is handled on the shell side. The downside is with multiple arguments you're gonna be constructing arrays of arguments on the shell side in order to pipe in. Not sure exactly how that would look, you could probably do some splatting.

PS > python -m hello world
Hello world!

PS > 'world', 'Mom', 'goodbye' | Invoke-Python -Module hello
Hello world!
Hello Mom!
Hello goodbye!

PS > Get-ChildItem -Filter *.md | Invoke-Python -Module hello
Hello C:\test.md
Hello C:\test2.md

The one-liner equivalent of this is 'world', 'Mom', 'goodbye' | ForEach-Object -Process {python -m hello $_}, but the below Invoke-Python helper function tucks some of that away.

Contents of hello.py

"""Say hello."""

import typer

def main(name: str = typer.Argument(...)):
    typer.echo(f"Hello {name}!")

if __name__ == "__main__":
    typer.run(main)

Contents of common.ps1

function Invoke-Python {
    <#.SYNOPSIS
    Invoke a Python module.
    #>

    Param(
        # Arbitrary input item.
        [Parameter(Mandatory, ValueFromPipeline)]$Item,

        # Python module to pass input to.
        [Parameter(Mandatory)]$Module
    )

    process {
        python -m $Module $Item
    }
}

celsiusnarhwal commented 1 year ago

Hi,

I found a solution to the problem I was working on. My automation script takes and string as CLI argument and writes it after couple of seconds.
python write_text.py "write"

# I converted it in the global script so I can run it from anywhere. Now, I use it like:
write_text "write"
However, there were cases where string I wanted to auto write was coming from some other bash command output so I wanted my script to run in both cases:
write_text "write"

# and, below example read from stdin

echo "write" | write_text

# Error if you don't pass CLI argument or don't provide stdin
This was challenging but thanks to this comment, I got an idea and here's example that let you pass argument and read from stdin.
import sys
import typer

def main(
    name: str = typer.Argument(
        ... if sys.stdin.isatty() else sys.stdin.read().strip()
    ),
):
    typer.echo(f"Hello {name}")

if __name__ == "__main__":
    typer.run(main)
I hope this will help you in resolving the issue

This doesn't work if you have more than one command using this pattern thanks to an inconvenient clash of the ways Python reads I/O streams and evaluates default function argument values. sys.stdin will be read — and its position moved to the end of the stream — as soon as the first command like this is defined, which means future commands that try to do this will read from an empty stream.

I'm currently working around this by using sentinel to indicate that a value should be read from standard input:

# This script is complete and should run as-is.

import sys

import sentinel
import typer

PIPE = sentinel.create("PIPE_FROM_STDIN")

def main(
    name: str = typer.Argument(
        ... if sys.stdin.isatty() else PIPE
    ),
):
    if name == str(PIPE):
        name = sys.stdin.read()

    typer.echo(f"Hello {name}")

if __name__ == "__main__":
    typer.run(main)

asmmartin commented 1 year ago

This looks like it works to me:

import sys

import typer
from typing_extensions import Annotated

def main(input_file: Annotated[typer.FileText, typer.Argument()] = sys.stdin):
    typer.echo(input_file.read())

if __name__ == '__main__':
    typer.run(main)

jankovicgd commented 4 months ago

This looks like it works to me:

import sys

import typer
from typing_extensions import Annotated

def main(input_file: Annotated[typer.FileText, typer.Argument()] = sys.stdin):
    typer.echo(input_file.read())

if __name__ == '__main__':
    typer.run(main)

This works, but then mypy complains with following Incompatible default for argument "input_file" (default has type "TextIO", argument has type "FileText")

fastapi / typer