PrefectHQ / prefect

Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
https://prefect.io
Apache License 2.0
15.86k stars 1.55k forks source link

Add REPL environment to Prefect CLI #9357

Open sugibuchi opened 1 year ago

sugibuchi commented 1 year ago

First check

Prefect Version

2.x

Describe the current behavior

Users need to execute Prefect CLI for every operation interacting with Prefect projects.

However, the bootstrapping of Prefect CLI is very slow, mainly due to the many Python modules loaded by the CLI (it takes several seconds). This slowness of Prefect CLI degrades the user experience during workflow development.

Describe the proposed behavior

It would be nice if Prefect CLI could provide a REPL environment allowing users to execute commands quickly without reload of Python modules.

Since Prefect CLI uses typer, a CLI framework based on click, we can easily integrate a REPL environment by using click-repl package.

# in prefect.cli.root
import click
import click_repl

@app.command()
def repl():
    """ Start REPL environment """
    click_repl.repl(click.get_current_context())

Example Use

Then you can launch a REPL environment by prefect repl.

$ prefect repl
> deploy --help
...

It takes several seconds to start a REPL environment. But once the REPL env starts, we can execute commands very quickly without loading Python modules every time. We can smoothly move in the command tree, show help and perform commands immediately. The overall UX in our test is pretty good and much more comfortable than running Prefect CLI in a terminal.

Some commands do not work as expected in this quick integration of REPL.

But it would significantly improve UX in Prefect flow development if Prefect CLI could officially provide a REPL environment.

Additional context

No response

zanieb commented 1 year ago

I think someone did this for Prefect 1 as well, perhaps we can use that as prior work I think it's likely we'd want a similar implementation.

I think there's an additional takeaway here that we should prioritize enhancing the speed of our CLI. One blocker is https://github.com/tiangolo/fastapi/issues/4644