--track-energy feature

[cappadokes]

Python is popular, even on resource-constrained devices. Energy is important. A nice-to-have feature would be a pyperf option that, if enabled, measures energy consumption instead of execution time. The only requirement for this pull request to bear meaning is the availability of some always-increasing measurement of the total energy spent by a device component. An example is the interface exposed by Intel's RAPL, which was used as the environment for this pull request.

Instructions for utilizing the --track-energy option are:

1. Write a dummy C library containing a single function that just reads from a file saved at the location denoted by the function's input parameter (a sample implementation is included as libreaden.so, the corresponding source file being read_file.c). Save the library in pyperf/pyperf.
2. Export the following environment variables:
   • LIBREADEN, containing the path to the C library.
   • ENFILE, containing the path to the energy measurement file.
3. Run your pyperf script with --track-energy and --inherit-environ LIBREADEN,ENFILE.

Measurements saved will now be energy instead of execution time.

The output will mention "seconds" instead of Joules or whatever. We haven't worked on the measurement units metadata yet.

[vstinner]

Run your pyperf script with --track-energy and --inherit-environ LIBREADEN,ENFILE.

--track-energy should imply --inherit-environ LIBREADEN,ENFILE.

[vstinner]

What is LIBREADEN? Where does it come from? Please write documentation.

[cappadokes]

What is LIBREADEN? Where does it come from? Please write documentation.

I am sorry, this is my first open-source contribution ever. I tried to be instructive in this PR's initial text. What qualifies as 'documentation' exactly? Do you mean editing the repo's README? Something else?

[corona10]

@cappadokes I love this feature but same as @vstinner comment, Can you please leave a link related to LIBREADEN, it would help to review this PR :)

[cappadokes]

@cappadokes I love this feature but same as @vstinner comment, Can you please leave a link related to LIBREADEN, it would help to review this PR :)

OK, I will do my best to expand below. It would be great if you could then direct me on what specific places of this repo other than our thread here this information should exist.

The present PR introduces the implementation of a simple energy consumption benchmarking feature. It was developed and tested on an Intel(R) Core(TM) i7-8750H x86_64 Linux machine for pyperf 2.3.1, with CPython 3.10.2.

Requirements

• A file-based interface exposing the system's aggregate energy consumption over time. Modern Intel CPUs provide such an interface via RAPL. On the example sysfs tree included in the previous link, you can see some files named energy_uj. These are single-line files containing an always-increasing value of the total microJoules consumed from the component since startup. There are many energy_uj files in many places of the intel-rapl tree, each one corresponding to a different scale of measurement (e.g. the whole package, the CPU, its caches, the DRAM).
• A dummy C shared library (dummy since it does one thing only) for accessing the interface described above. This format was chosen so as to a) be as fast as possible, compared for instance to reading a file through Python and b) be agnostic with regard to the specific interface employed by the machine. An example implementation is read_file.c, which I include to this PR. Once the user has compiled such a library, she should save it in pyperf/pyperf (I used ctypes.CDLL to invoke it, yet placing the .so in arbitrary spots and calling its contents proved non-trivial for my skills).

Usage

export ENFILE=            #    Absolute path to energy_uj or equivalent resource
export LIBREADEN=    #    Absolute path to pyperf/pyperf/dummy_c_library.so
python3 pyperf_using_benchmark.py --track-energy --inherit-environ ENFILE,LIBREADEN

Motivation I discovered pyperf through pyperformance, since I wanted to do some legit energy-related experiments on real-world benchmarks. For convenience (and since --inherit-environ is exposed yet broken there) I have also created a PR there in order to integrate it all together.

First experiences I have produced sets of ~20 stable pyperformance benchmark results for my machine's CPU and DRAM energy consumption while using the --rigorous option. It's a small but non-negligible number compared to the ~60 total cases in pyperformance. As I'm writing this I realized that my experiments did not use the --system-tune feature, so a new batch might prove even more productive.

Limitation The only thing not implemented yet is the measurement units metadata. Thus everything is currently reported as seconds instead of microJoules (the unit should however depend on the energy interface. So hard-coding microjoules is not the way to go). But dealing with this detail looks neither urgent nor challenging for now.

[vstinner]

The documentation source lives in the doc/ subdirectory and is rendered at: https://pyperf.readthedocs.io/

[cappadokes]

The documentation source lives in the doc/ subdirectory and is rendered at: https://pyperf.readthedocs.io/

And should I directly edit the .rst files in doc/ ? Are they a result of some automatic docs generation utility (and thus require that I edit the .py source code instead of the .rst's)? Again, sorry for the spamming, but dealing with these from the get-go will save time from all of us.

[cappadokes]

Hi all. I have now updated the documentation so as to include --track-energy. I have started the analogous PR on the pyperformance side: https://github.com/python/pyperformance/pull/140

There is nothing more to be done at this point.