jazzband / prettytable

Display tabular data in a visually appealing ASCII table format
https://pypi.org/project/PrettyTable/
Other
1.32k stars 149 forks source link

Speedup: lazy imports and remove import #276

Closed hugovk closed 5 months ago

hugovk commented 5 months ago

Command-line interfaces value speed, and unused imports can slow things down a lot.

We can use python -X importtime and tuna to identify bottlenecks: https://medium.com/alan/how-we-improved-our-python-backend-start-up-time-2c33cd4873c8

For example:

python3.13 -X importtime -c 'import prettytable' 2> import.log
tuna import.log

We're using Python 3.13.0a3 here, because it already includes some optimisations of its own.

Baseline

Starting on main, prettytable: 0.027 s (87.8%)

image

And let's begin by zooming into prettytable.prettytable: 0.010s (32.8%)

image

Lazy import json

json is a good first target. Taking 0.001 s (2.7%), it's only used by one function to load from JSON, and another to save. If doing plain text tables (or HTML etc), you won't use them. Let's move the import into those two functions where it's actually needed:

image

Result: prettytable.prettytable: 0.009 s (30.3%)

Lazy import random

random is only used by a function to set random styles, and is mostly for fun. Lazy import it to save 0.001 s (1.8%):

image

Result: prettytable.prettytable 0.009 s (28.5%)

Lazy import csv

Like json, csv is only used for loading or saving CSV and takes 0.003 s (10.8%).

image

Result: prettytable.prettytable 0.008 s (27.1%)

Lazy import copy

Used in three places: a function to return a copy of a table. Is this used much? And also in other functions, themselves used when getting a number of string, HTML, CSV, JSON, Latex versions of a table. That's quite a few, so will likely end up being imported anyway, but it's a simple change and we may save 0.001 s (2.7%) in a few places.

image

Result: prettytable.prettytable 0.008 s (25.9%)

Lazy import textwrap

Only used in one function when getting a string version of a table. I think a common operation, but again, a quick saving for those cases when not.

image

Result: prettytable.prettytable 0.007 s (24.6%)

Lazy import wcwidth

Called by one function, but itself called several times, mostly for getting string tables? Well, saving of 0.001 s (2.2%) if not needed.

Result: prettytable.prettytable 0.007 s (23.0%)

Lazy import html.escape

Only used when getting HTML versions of tables: 0.001 s (2.3%)

image

Result: prettytable.prettytable 0.007 s (22.5%)

Replace math.floor with int

For positive inputs math.floor and int give the same results, we can avoid importing math to save a (rounded) 0.000 s (0.9%). We were calling int on the result of math.floor anyway!

image

Result: prettytable.prettytable 0.006 s (21.6%)

Lazy import importlib.metadata

Finally, let's step out back to prettytable:

image

importlib.metadata takes a big chunk of the original: 0.015 s (50.9%)!

We only use this to set __version__ on the off-chance someone might access it:

import importlib.metadata

__version__ = importlib.metadata.version(__name__)

We can instead only import when __version__ is accessed:

def __getattr__(name: str) -> Any:
    if name == "__version__":
        import importlib.metadata

        return importlib.metadata.version(__name__)

    msg = f"module '{__name__}' has no attribute '{name}'"
    raise AttributeError(msg)
image

Result: prettytable 0.007 s (65.1%).


Compare with our original prettytable: 0.027 s (87.8%), that's a big saving!

This is an idealised result and in reality a few of these will be imported, based on the use case. Still, those that do apply will help CLIs.

A quick look at the remaining "big" ones:

But we've achieved enough here for now!

codecov[bot] commented 5 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (80da6be) 94.21% compared to head (6279c94) 94.38%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #276 +/- ## ========================================== + Coverage 94.21% 94.38% +0.16% ========================================== Files 5 5 Lines 2387 2405 +18 ========================================== + Hits 2249 2270 +21 + Misses 138 135 -3 ``` | [Flag](https://app.codecov.io/gh/jazzband/prettytable/pull/276/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jazzband) | Coverage Δ | | |---|---|---| | [macos-latest](https://app.codecov.io/gh/jazzband/prettytable/pull/276/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jazzband) | `94.34% <100.00%> (+0.16%)` | :arrow_up: | | [ubuntu-latest](https://app.codecov.io/gh/jazzband/prettytable/pull/276/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jazzband) | `94.34% <100.00%> (+0.16%)` | :arrow_up: | | [windows-latest](https://app.codecov.io/gh/jazzband/prettytable/pull/276/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jazzband) | `94.30% <100.00%> (+0.16%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jazzband#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.