tbarbette / npf

Network Performance Framework: easy-to-use experiment manager with automated testing, result collection, and graphing
GNU General Public License v3.0
40 stars 15 forks source link

Multiple pyexit #30

Closed MassimoGirondi closed 3 months ago

MassimoGirondi commented 2 years ago

This is not completely finished yet, but let's start tracking the pending changes.

One may want to have multiple pyexit sections that are called when importing modules or when certain tags are given. For instance, one may have a post processing pyexit that should be executed only some times. It was not possible to do that in NPF (AFAIK).

Also, includes late_variables when running pyexit even in imported modules, otherwise it is not possible to access them. Pyexits are now executed in alphabetical order so one can have for instance "00-preprocess", ..., "99-averages", where in the last it calculated all the averages of the variables created/changed in the previous steps (useful if you want e.g. percentiles of time metrics). This functionality is not completely tested yet.

Finally, an initial version of a VIM syntax highlight for npf files.

tbarbette commented 2 years ago

"Finally, an initial version of a VIM syntax highlight for npf files." +1000 !

Are the imports stuff actually run?

MassimoGirondi commented 2 years ago

See the attached screenshot. When running with the upstream version (top) the pyexit in the module is ignored, while with this patch it is executed. The import script is executed in both cases (see the -MODULE results). image

The code executed in the screenshot is the one in the integration/imports folder, with the pyexit names stripped down to be executed with the upstream npf:

main.npf:


%config
default_repo=local //No program under test
accept_zero={LOG} //Log may be "0" that is a valid number. By default NPF consider a test as failed is a value is 0
var_unit={RESULT: }
var_format={RESULT:%d}

%variables
N=[1-10]

%script
echo "RESULT-N $N"
echo "RESULT-LOG $(( log($N) ))"
echo "RESULT-EXP $(( pow(2,$N) ))"

%import module

%pyexit
print("RESULT_MAIN_PYEXIT")

module.npf

%script
echo "RESULT-N $N"
echo "RESULT-LOG-MODULE $(( log(2*$N) ))"
echo "RESULT-EXP-MODULE $(( pow(3,$N) ))"

%pyexit
print("This is pyexit in module!")
RESULTS["PYEXIT_MODULE_RESULT1"] = 7
tbarbette commented 2 years ago

Just fix the CI and then I'll merge :)

tbarbette commented 3 months ago

Rebasing in new PR