Open jaraco opened 7 months ago
Have you had time to think more about this? I am very interested!
I haven't yet. If/when I do, I'll update here. If you're interested in exploring the landscape, let me know and I can give you a tour of this project and how I might go about it. Basically, it boils down to creating "exercises" (Python-syntax functions that perform the warmup and benchmark logic) that are somehow customized for checking execution time at import (probably python -m importtime
as described here). Whether there should be an explicit decorator for indicating to check import time or if it should be inferred from the structure of the benchmark logic, I'm not yet sure. I could imagine the test to look something like:
def import_time_perf():
import importlib_metadata
And pytest-perf could infer that because the only statement in the test is an import statement, it should use the "import time" check.
Or it could be explicit:
from pytest_perf.deco import import_time
@import_time('importlib_metadata')
def import_time_perf():
import importlib_metadata
But that doesn't feel quite right, because "importlib_metadata" has to be specified twice.
Maybe instead, since currently the presence of perf
in the function name is used to check performance, maybe _import_time
could be used to detect import time:
def check_import_time():
import importlib_metadata
I think I'm liking that approach best.
In https://github.com/python/cpython/pull/114664, I was experimenting with using
pytest-perf
to capture the import latency ofimportlib_metadata
but learned that timeit may not support that use-case readily.Let's explore if there's something pytest-perf can do to make measurement of import latency straightforward.