reframe-hpc / reframe

A powerful Python framework for writing and running portable regression tests and benchmarks for HPC systems.
https://reframe-hpc.readthedocs.org
BSD 3-Clause "New" or "Revised" License
214 stars 101 forks source link

[feat] Enable default value type checking and implicit conversions at the variable declaration level #3115

Closed vkarak closed 6 months ago

vkarak commented 6 months ago

Currently, the default values of variables or the values assigned to them in the test class body are not type checked and no implicit conversions are applied to the value being assigned. This is deferred at the class instantiation time where the default values of variables are injected to the newly created test object and type checked through the associated descriptor.

This PR moves the type checking and implicit conversions, i.e., the functionality of the descriptors at the class level. We do so, with two tricks:

  1. We extend the Python descriptor model by having __set__() return the value that the descriptor would write into the object if the object passed to __set__() is None.
  2. We anticipate the call to the descriptor's __set_name__() at the same time that this function is called for the variable. And it is at this point that we also call the descriptor's __set__() method to force the type checking and any type conversion. For type conversions we do respect the allow_implicit argument of variables.

The advantages of this method are

Todos

codecov[bot] commented 6 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (e7e93f0) 86.67% compared to head (317fef3) 86.66%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## develop #3115 +/- ## =========================================== - Coverage 86.67% 86.66% -0.02% =========================================== Files 61 61 Lines 12082 12092 +10 =========================================== + Hits 10472 10479 +7 - Misses 1610 1613 +3 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

vkarak commented 6 months ago

The test instantiation time increases by about 5% and the overall time of the following benchmark increases by 3.5%:

reframe -c unittests/resources/checks/hellocheck.py -n ^HelloTest --repeat=1000 -l

I believe the difference comes from the fact that we now validate every assignment of a variable, whereas before we were validating once during test instantiation.