Perfexionists / perun

Lightweight Performance Control System
GNU General Public License v3.0
16 stars 14 forks source link

[Perun Import] Add support to read input relative to user supplied directory #251

Closed jirka-h closed 3 months ago

jirka-h commented 3 months ago

Hi,

we will create "generic Profile Stats file" - see issue #250 - relative to some directory. We need support in perun import to specify this directory:

perun import --input-directory DIR

Read input files relative to directory DIR

Example: 
CSV file is in /mnt/data/www/testing/sched/results/stress-ng/RHEL-10.0-20240722.29/gold-1s.tpb.lab.eng.brq.redhat.com/kernel-6.10.0-15.el10.x86_64/4b396157-8d3e-45e6-9b13-5ae87ed429b3/PROCESSED_DATA` directory

CSV file looks like this:
#Stackfile,Exit_code,bogo-ops-per-second-real-time|bogo-ops-per-second|higher_is_better
Run1/kill-threadsNO1_perf_record.perf-folded.gz,0,18511.379883
Run2/kill-threadsNO1_perf_record.perf-folded.gz,0,16463.042429
Run3/kill-threadsNO1_perf_record.perf-folded.gz,0,27043.820382
Run4/kill-threadsNO1_perf_record.perf-folded.gz,0,16562.478509
Run5/kill-threadsNO1_perf_record.perf-folded.gz,0,18575.145957

We will run (pseudocode) perun in temporary directory created by mktemp -d
/tmp/tmp.CyZAHxyvNf
perun import --input-directory  /mnt/data/www/testing/sched/results/stress-ng/RHEL-10.0-20240722.29/gold-1s.tpb.lab.eng.brq.redhat.com/kernel-6.10.0-15.el10.x86_64/4b396157-8d3e-45e6-9b13-5ae87ed429b3/PROCESSED_DATA --csv perun_input.csv

With this input, perun will look for input files with the full path being: DIR/csv_path 

/mnt/data/www/testing/sched/results/stress-ng/RHEL-10.0-20240722.29/gold-1s.tpb.lab.eng.brq.redhat.com/kernel-6.10.0-15.el10.x86_64/4b396157-8d3e-45e6-9b13-5ae87ed429b3/PROCESSED_DATA/Run1/kill-threadsNO1_perf_record.perf-folded.gz

Thanks a lot! Jirka

JiriPavela commented 3 months ago

Sure, this sounds reasonable, I'll add another option to the perun import command.

jirka-h commented 3 months ago

Awesome, thank you!