PrefectHQ / prefect

Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
https://prefect.io
Apache License 2.0
15.82k stars 1.55k forks source link

Prefect CLI: Add a batch mode / plain output feature to the command line interface #6930

Open dcupp opened 1 year ago

dcupp commented 1 year ago

First check

Prefect Version

2.x

Describe the current behavior

The prefect command line interface outputs information in a way that makes it difficult to use with commands like grep, sed and awk.

If I run a command like prefect deployment ls, the command outputs two columns of information, but:

  1. the output has lots of non-whitespace characters that aren't data, being used for graphics (9473, 9523) -- this makes it more challenging to process
  2. there are header lines, and other extra lines (horizontal dividers) with no information -- lines like these are particularly awkward to deal with when piping commands
  3. the CLI command detects the width of my terminal, and if the lengths of the output rows exceed my terminal width, it truncates the data, and I don't get the output anyway.

Describe the proposed behavior

It would be great if there was an ENV var, or preferably a command line flag, that would turn off all of the fancy output, color codes, graphical characters, header/divider rows, and line truncation.

Example Use

I think a short flag would be best, like -b for batch mode or -q for quiet:

prefect deployment ls -b

Or an env var:

PLAIN_OUTPUT=1 prefect deployment ls

Additional context

I can provide an example use case: deleting deployments with a certain name, something like:

prefect deployment ls | grep "foo.*bar" | xargs -n 1 prefect deployment delete

There may be a better way to delete deployments that start and end with a certain string; I just made that up as an example.

zanieb commented 1 year ago

We'd like to have a --json flag for all CLI commands which should cover this by outputting content in parsable JSON. I'd prefer the implementation be reusable across all of our CLI commands somehow. This will require some refactoring but I'm not opposed to a community contribution if it's done well.

dcupp commented 1 year ago

A --json flag sounds like a great idea but doesn't really help my use case. I think filtering everything through jq first would be more onerous than writing my own CLI client.

[EDIT] If you did that, maybe a more generic --output-format json|yaml|hocon|csv would be helpful