search

Perfexionists / perun

Lightweight Performance Control System
GNU General Public License v3.0
16 stars 14 forks source link

Add support for lazy loading and imports of some expensive subpackages and modules to speed up Perun startup time #259

Closed JiriPavela closed 1 month ago

JiriPavela commented 1 month ago

This PR implements the mechanism proposed in #223 for the most expensive modules that were being imported during perun --version, perun import, and perun showdiff.

JiriPavela commented 1 month ago
  1. If I am looking right, view is not fully lazy loaded and it was quite a source of the problems (with holoviews/bokeh being quite huge). Is that WIP?

Right now, view modules are being lazy loaded the naive way, i.e., by local imports in lazy_get_cli_commands. As this PR is meant to be more of a hotfix, I didn't touch code that already achieved the same goal, albeit in another way. Subsequent PRs will aim to port the entire codebase to lazy_loader approach.

  1. Can you show some numbers? Like running perun help, perun status, perun init and maybe perun showdiff before and after? Just few runs and few numbers so we can compare how much we have saved, because it did come with a cost (worse readability, bigger complexity, and I fear it might be harder for IDEs to suggest).

I agree with the worse readability and bigger complexity. That is sadly the price of Python not having a native support for lazy loading. IDE, intellisense or autocomplete should not be affected, that's what the __getattr__, __dir__, __all__ = lazy.attach_stub(... in __init__.py files is there for. Nonetheless, some IDEs still run into trouble with it, but imports in the form of from perun import check as check usually solve it (see similar bugs with some other packages), although pylint complains about such import statements.

As for the numbers, I measured five runs of different commands and chose the median values: