I had the idea of adding a custom package-wide profiling utility based on a singleton that is imported in all modules that we want to profile. Unfortunately i have not found anything that does this already, but it is also not so hard to implement. I think it's obvious that this could give us several advantages over print statements in the whole code:
Direct access to machine-readable timing reports without having to parse the output from print statements (we can export timing reports as CSV, JSON, or even as DataFrame)
We could easily switch the profiler on or off with an environment variable.
Just add a decorator to methods/functions that should appear in the timing report without cluttering the method body
I already put together a bare-bones version of this in this gist.
We could follow the same strategy for logging, switching the code into debug mode with some simple flag. This can probably also be done with Python's built-in logging module.
I had the idea of adding a custom package-wide profiling utility based on a singleton that is imported in all modules that we want to profile. Unfortunately i have not found anything that does this already, but it is also not so hard to implement. I think it's obvious that this could give us several advantages over
printstatements in the whole code:printstatements (we can export timing reports as CSV, JSON, or even as DataFrame)I already put together a bare-bones version of this in this gist.
We could follow the same strategy for logging, switching the code into debug mode with some simple flag. This can probably also be done with Python's built-in
loggingmodule.Any thoughts on this?