search

fjebaker / SPEChpc-reframe

A reframe wrapper for SPEChpc, with a focus on measuring energy usage via perf events.
MIT License
0 stars 2 forks source link

Introduce cooldown period #6

Closed fjebaker closed 7 months ago

fjebaker commented 7 months ago

Dom mentioned there should be a cooldown period between runs, since the BMCs are only really accurate to within 30 seconds.

Ideally we let a node rest for about 30 seconds or so before queuing up the next job so that our time windows don't overlap, which could spoil the energy estimate.

fjebaker commented 7 months ago

As I doubt there will be a reframe option that will be right for this, one thing we could do is have a @run_before("run") hook which asks the scheduler when the last job on node X finished, and sleeps a given offset before letting reframe continue?

fjebaker commented 7 months ago

The above wont work since multiple jobs might get queued up to the scheduler simultaneously.

A different fix would be to add a sleep call at the end of each script, and then subtract the same amount from the database query we make?