reframe-hpc / reframe

A powerful Python framework for writing and running portable regression tests and benchmarks for HPC systems.
https://reframe-hpc.readthedocs.org
BSD 3-Clause "New" or "Revised" License
214 stars 102 forks source link

Fixed pipeline hook execution order may break existing workaround code #3012

Closed vkarak closed 9 months ago

vkarak commented 11 months ago

The fix #2997 may break existing code used to workaround this limitation in previous versions. More specifically, the following example will fail to run with reframe >= 4.3.4

class my_base_test(...):
    x = variable(int)

    @run_before('run')
    def use_x(self):
        y = self.x

@rfm.simple_test
class my_test(my_base_test):
    @run_before('run')
    def set_x(self):
        self.x = 1

    # Possible workaround for reframe <= 4.3.3
    @run_before('run')
    def use_x(self):
        super().use_x()

Affected versions: >= 4.3.4

How to fix

If you are using reframe >= 4.4, you should pin the base class hook to run last it its stage:

class my_base_test(...):
    x = variable(int)

    @run_before('run', always_last=True)
    def use_x(self):
        y = self.x

@rfm.simple_test
class my_test(my_base_test):
    @run_before('run')
    def set_x(self):
        self.x = 1

For reframe 4.3.4 specifically, where the always_last argument is not available, you should rewrite the initial workaround as

class my_base_test(...):
    x = variable(int)

    @run_before('run')
    def use_x(self):
        y = self.x

@rfm.simple_test
class my_test(my_base_test):
    @run_before('run')
    def use_x(self):
        self.x = 1
        super().use_x()

We should add a note about this in the docs and also review the current docs/tutorials on their hook syntax.