ewels / rich-click

Format click help output nicely with rich.
https://ewels.github.io/rich-click/
MIT License
583 stars 33 forks source link

WIP: Add blog #184

Closed ewels closed 2 months ago

ewels commented 3 months ago

Added a blog to the docs website.

Now just need to write a blog article!

dwreeves commented 3 months ago

For the purposes of showing the impact of lazy-loading Rich, let's include this, generated via the memory-profiler Python package:

image

The change sees memory usage go down by about 5 MB. (Explanation: rich_click.rich_help_rendering is itself lazy loaded, but eagerly loads rich modules, so the delta between import rich_click and import rich_click.rich_help_rendering shows the memory savings.) Honestly really good for a quick and simple internal refactor!

ewels commented 2 months ago

Nice! Do we care about time, or is it just memory consumption?

If it's the latter, how about making it a stacked bar plot?

Could have three bars:

That'd show nicely how much rich-click adds, and also show the difference between 1.7 and 1.8...

dwreeves commented 2 months ago

We can do time, and we can compare the versions for sure. I can whip that up.

I have commitments to update https://github.com/codeforboston/flagging this weekend so we'll see what I can sneak in.

dwreeves commented 2 months ago

I wrote the following for the Live Style Editor section:


### Live Style Editor

The coolest addition allowed by this change to full documentation is the [Live Style Editor](https://ewels.github.io/rich-click/editor).

We built this to address a problem we noticed, which is that the vast majority of users rely on the default **rich-click** styles.
Although we do think **rich-click**'s defaults are pretty good, it pointed to a potential developer experience issue that so many CLIs relied on the defaults.
We hope that the live style editor makes it easier for users to make style changes and to personalize their CLIs. 😁

Here is an example of a style that Phil made with the style editor:

[todo]

And here's one that Daniel made:

[todo]

Note the following:

Here is an example of a style that Phil made with the style editor:

It's cheesy, but I think it would be fun! πŸ˜… Can you give me something I can put there? πŸ₯ΊπŸ™ Just copy paste the output code into a Github comment and I'll do the rest of rendering it into the blog.

dwreeves commented 2 months ago

Profiling!

Execution time:

image

Memory:

image


What I'm running for my diagnostics:

#!/bin/bash
set -eo pipefail

export VIRTUAL_ENV=.venv_benchmarks
export PY_VERSION=3.12

speed_trials=40
mprof_trials=10

###############################################################################

uv venv "${VIRTUAL_ENV}" --python "${PY_VERSION}"

cat <<EOF > hello_click.py
import click

@click.command()
@click.option("--name", default="World", help="Name to greet.")
def hello(name):
    """Greet someone."""
    print(f"Hello, {name}!")
    if name == "Daniel":
        import time
        time.sleep(0.2)

if __name__ == "__main__":
    hello()
EOF

cat <<EOF > hello_rich_click.py
import rich_click as click

@click.command()
@click.option("--name", default="World", help="Name to greet.")
def hello(name):
    """Greet someone."""
    print(f"Hello, {name}!")
    if name == "Daniel":
        import time
        time.sleep(0.2)

if __name__ == "__main__":
    hello()
EOF

cat <<EOF > hello_argparse.py
import argparse

def main():
    parser = argparse.ArgumentParser(description="Greet someone.")
    parser.add_argument("--name", default="World", help="Name to greet.")
    args = parser.parse_args()
    print(f"Hello, {args.name}!")
    if args.name == "Daniel":
        import time
        time.sleep(0.2)

if __name__ == '__main__':
    main()
EOF

chmod +x hello_argparse.py
chmod +x hello_click.py
chmod +x hello_rich_click.py

################################################################################

function get_times {
  total_time=0
  filename="${1}"
  clear_pyc_files="${2}"

  # Run once to compile pyc files
  "${VIRTUAL_ENV}/bin/python" "${filename}" --name Phil >/dev/null

  for (( i=0; i < speed_trials; i++ ))
  do

    if [ "${clear_pyc_files}" = "true" ]; then
      find "${VIRTUAL_ENV}/lib/python${PY_VERSION}/site-packages/" -name '*.pyc' -delete
    fi

    start_time=$(gdate +%s.%N)
    "${VIRTUAL_ENV}/bin/python" "${filename}" --name Phil >/dev/null
    end_time=$(gdate +%s.%N)

    elapsed=$(echo "$end_time - $start_time" | bc)
    total_time=$(echo "$total_time + $elapsed" | bc)

  done

  average_time=$(echo "$total_time / $speed_trials" | bc -l)
  echo "Average time for ${filename} with clear_pyc_files=${clear_pyc_files}: $average_time seconds"

}

function get_mprof {
  total_mib=0
  filename="${1}"
  clear_pyc_files="${2}"

  # Run once to compile pyc files
  "${VIRTUAL_ENV}/bin/python" "${filename}" --name Phil >/dev/null

  for (( i=0; i < mprof_trials; i++ ))
  do

    if [ "${clear_pyc_files}" = "true" ]; then
      find "${VIRTUAL_ENV}/lib/python${PY_VERSION}/site-packages/" -name '*.pyc' -delete
    fi

    "${VIRTUAL_ENV}/bin/mprof" run "${VIRTUAL_ENV}/bin/python" "${filename}" --name Daniel >/dev/null
    output=$("${VIRTUAL_ENV}/bin/mprof" peak)
    echo $output
    mprof_file=$(echo "$output" | grep 'mprofile' | awk '{print $1}')
    memory_usage=$(echo "$output" | grep 'mprofile' | awk '{print $2}' | bc)
    total_time=$(echo "$total_mib + $memory_usage" | bc)
    rm "${mprof_file}"
  done

  average_memory_usage=$(echo "$memory_usage / $mprof_trials" | bc -l)
  echo "Average MiB consumed for ${filename} with clear_pyc_files=${clear_pyc_files}: $memory_usage MiB"

}

################################################################################

# Times

uv pip install --no-binary :all: "rich-click==1.8.0dev7"

get_times hello_argparse.py true
get_times hello_click.py true
get_times hello_rich_click.py true

get_times hello_argparse.py false
get_times hello_click.py false
get_times hello_rich_click.py false

uv pip install --no-binary :all: "rich-click==1.7"

get_times hello_rich_click.py true
get_times hello_rich_click.py false

################################################################################

# Memory profiling

uv pip install memory-profiler
uv pip install --no-binary :all: "rich-click==1.8.0dev7"

get_mprof hello_argparse.py true
get_mprof hello_click.py true
get_mprof hello_rich_click.py true

get_mprof hello_argparse.py false
get_mprof hello_click.py false
get_mprof hello_rich_click.py false

uv pip install --no-binary :all: "rich-click==1.7"

get_mprof hello_rich_click.py true
get_mprof hello_rich_click.py false
dwreeves commented 2 months ago

Added typer, yay or nay?

image

image

If we add Typer, I do think it is worth clarifying a few caveats as a courtesy, since the intent would be to provide context for what a reasonable baseline is for a Click wrapper, not to start a war over CLIs frameworks.

Something like this:

We include Typer in our profiling to show a reasonable baseline for a Click wrapper's overhead.
Typer is an ambitious and great project that's doing quite a bit under the hood, and it's reasonable to expect it to take a little more time and memory.

What is a little less reasonable is how **rich-click** 1.7 left a few free optimizations on the table:

1. Only import `rich` when rendering help text.
2. Use `click.__version__` instead of `importlib.metadata.version("click")` for Click 7 compat.

Combined, these two changes account for all the performance improvements.

Performance isn't everything; if it was, we'd all be using `argparse`, or we'd abandon Python altogether for Rust.
This is also peanuts in the grand scheme of things.
In all likelihood, you've spent more time reading this section than the cumulative amount of time you'll save by `pip install --upgrade`-ing your **rich-click** 1.7 project.
(There are other reasons to upgrade to 1.8 than performance, of course!)

So why bother improving **rich-click**'s performance if it's not a big deal?
Because we're honored every time someone chooses **rich-click** for their applications, and we want to pay it back by keeping things as efficient as we reasonably can.
Your application is complex and special and all yours.
We're excited we get to be a very small part of what you're doing, 🫢 and we'll do our best to keep our end of things neat and tidy.
dwreeves commented 2 months ago

Did something happen with mypy in the last 2 days? All I did was merge this in with main, which passed 2 days ago. πŸ€” Hm!

ewels commented 2 months ago

Looking great!

Do we need with / without bytecode? If we keep it then I think it needs some explanation in the text as to what it means.

The only thing that the bar plots miss is the usage of rich_click when not generating help text. I was thinking that this is one of the main improvements with the lazy loading..

But yeah, basically ready to merge all of this and release now from my side πŸ‘πŸ»

Phil

dwreeves commented 2 months ago

Do we need with / without bytecode? If we keep it then I think it needs some explanation in the text as to what it means.

I'll add a quick explanation, something like this?:

!!! note
    Python regularly compiles `.py` files into `.pyc` files to speed up code execution. The **with bytecode** metrics measure performance _with_ these `.pyc` files, and the **without bytecode** metrics measure performance _without_ them.

I don't mind including them. If someone doesn't know what that is, and they learn what it is from this blog post, then it means the blog post had additional educational value other than just promoting rich-click 1.8. I think that's a good thing! πŸ˜„

The only thing that the bar plots miss is the usage of rich_click when not generating help text. I was thinking that this is one of the main improvements with the lazy loading..

That's what it's testing and what I mean by "during command execution". I realize that's not clear, so I'll make sure it is!

During command execution (i.e. when running your app, as opposed to generating help text), **rich-click** now loads faster and takes up less memory than before:
dwreeves commented 2 months ago

Speaking of educational value, I also added a brief explanation of what we're doing:

Why is **rich-click** 1.8 more performant? 1.7 left a few free optimizations on the table:

1. Only import `rich` when rendering help text.
2. Use `click.__version__` instead of `importlib.metadata.version("click")` for Click 7 compat.

For the first change, this meant replacing code like this...:

```python
from typing import IO, Optional

from rich.console import Console

from rich_click.rich_help_configuration import RichHelpConfiguration

def create_console(config: RichHelpConfiguration, file: Optional[IO[str]] = None) -> Console:
    console = Console(
        # ...
    )
    return console

...with code like this...:

from typing import TYPE_CHECKING, IO, Optional

from rich_click.rich_help_configuration import RichHelpConfiguration

if TYPE_CHECKING:
    from rich.console import Console

def create_console(config: RichHelpConfiguration, file: Optional[IO[str]] = None) -> "Console":
    from rich.console import Console
    console = Console(
        # ...
    )
    return console

...so that Rich is only loaded when it is needed!



I personally find content like this to be more engaging when people tell you their tricks, instead of just promoting new features. Some readers may not know what `typing.TYPE_CHECKING` is or how it can be used, and this would be valuable educational content for them.