Notice: We plan to share a new implementation of PerfSpect in the coming months. The new implementation of PerfSpect will enhance its functionality by offering "live" metric generation. This means that the current post-processing step will be optional. Additionally, PerfSpect will incorporate features from other Intel tools, streamlining their acquisition and deployment for users. Stay tuned!
PerfSpect is a system performance characterization tool built on top of linux perf. It contains two parts:
perf-collect: Collects hardware events at a 5 second output interval with practically zero overhead since PMU's run in counting mode.
sudo ./perf-collect
default system widesudo ./perf-collect --socket
sudo ./perf-collect --cpu
sudo ./perf-collect --pid <process-id>
sudo ./perf-collect --cid
_by default, selects the 5 containers using the most CPU at start of perf-collect. To monitor specific containers provide up to 5 comma separated cids i.e. sudo ./perf-collect
default run until terminatedsudo ./perf-collect --timeout 10
run for 10 secondssudo ./perf-collect --app "myapp.sh myparameter"
runs for duration of another processperf-postprocess: Calculates high level metrics from hardware events
./perf-postprocess
wget -qO- https://github.com/intel/PerfSpect/releases/latest/download/perfspect.tgz | tar xvz
cd perfspect
sudo ./perf-collect --timeout 10
./perf-postprocess
As seen in the examples above, sudo
is the standard approach to running perf-collect with elevated privileges. If sudo
is not possible and running as the root user is not possible, then a user may request the following changes be made to the system by an administrator:
for i in $(find /sys/devices -name perf_event_mux_interval_ms); do echo 125 > $i; done
Recommend returning these settings to their prior values when analysis with PerfSpect is complete.
perf-collect outputs:
perfstat.csv
: raw event counts with system metadataperf-postprocess outputs:
metric_out.sys.average.csv
: average metricsmetric_out.sys.csv
: metric values at every 5 second intervalmetric_out.html
: html view of a few select metricsModify the template deamonset.yml to deploy in kubernetes
perf - PerfSpect uses the Linux perf tool to collect PMU counters
Different events require different minimum kernels (PerfSpect will automatically collect only supported events)
Requires recent python. On successful build, binaries will be created in dist
folder
pip3 install -r requirements.txt
make
Note: Most metrics and events come from perfmon and TMA v4.5