etmc / tmLQCD

tmLQCD is a freely available software suite providing a set of tools to be used in lattice QCD simulations. This is mainly a HMC implementation (including PHMC and RHMC) for Wilson, Wilson Clover and Wilson twisted mass fermions and inverter for different versions of the Dirac operator. The code is fully parallelised and ships with optimisations for various modern architectures, such as commodity PC clusters and the Blue Gene family.
http://www.itkp.uni-bonn.de/~urbach/software.html
GNU General Public License v3.0
32 stars 47 forks source link

adding in which monomial the timing is measured #505

Closed Marcogarofalo closed 2 years ago

Marcogarofalo commented 2 years ago

I introduce a global char * which is set to the working monomial in the update

kostrzewa commented 2 years ago

I would prefer not to have yet another global variable to take care of. I think it would be better to add arguments to tm_stopwatch_push, such that it can also print at some debug level (let's say >= 3).

Marcogarofalo commented 2 years ago

Then how tm_stopwatch_push should know the monomial? The other option that I can think of is to pass the monomial name to all the functions like sw_spinor_eo, sw_all, deriv_Sb, ... Do you have something better?

kostrzewa commented 2 years ago

It doesn't need to. The only thing that's required is that inside the monomial functions, one calls:

tm_stopwatch_push(&g_timers, 0, 3, mnl->name, __func__);

which then prints something like

# monomial_name: Entering __func__ level: %d

In the analysis of the output, the timings can then be associated to the correct context by reading monomial_name and the level and the final output of:

tm_stopwatch_pop(&g_timers, 0, 1, mnl->name, __func__);

Anything at levels higher than these two outputs belongs to that context.

kostrzewa commented 2 years ago

Another option is to add an explicit "context" information for tm_stopwatch_push. In other words, the g_timers struct contains another array of strings (with some good length each, up to 500 or so).

tm_stopwatch_push(&g_timers, "context");

which then stores "context" in g_timers.context[level]. This context could be, for example, mnl->name and can even be used recursively in the logic (if the argument is empty, use the context of the previous level).

tm_stopwatch_push(&g_timers, "")

would thus result (internally) in

g_timers.context[level] = g_timers.context[level-1];

And then you can even have the context output at every call.

Marcogarofalo commented 2 years ago

ok, then in a function like

void deriv_Sb(const int ieo, spinor * const l, spinor * const k,
              hamiltonian_field_t * const hf, const double factor) 

I have to add

void deriv_Sb(const int ieo, spinor * const l, spinor * const k,
              hamiltonian_field_t * const hf, const double factor, const char* mnl_name ) 
kostrzewa commented 2 years ago

I have to add

No, it's not necessary. When you have:

Entering xxx in yyy, level: n

in the output, anything above this level is automatically within that context. In other words: the deriv_Sb used within the monomial function will automatically have level: n+1 and can be associated correctly at the analysis stage.

In addition, doing what I wrote above in https://github.com/etmc/tmLQCD/pull/505#issuecomment-918173628 would even give you the context locally.

kostrzewa commented 2 years ago

To be more concrete, you could have:

tm_stopwatch_push(&g_timers, mnl->name);

at the top of det_derivative, for example. Internally, this sets

timers->context[level] = "name of the monomial";

Then, when tm_stopwatch_push(&g_timers, "") is called with an empty context (for example, inside deriv_Sb), this instead sets

timers->context[level] = timers->context[level-1];

which can then be printed as part of the tm_stopwatch_pop inside of the function is question. This way all pops would be instrumented and one would only provide context when this makes sense from a design and/or logical pov.

kostrzewa commented 2 years ago

@Marcogarofalo can you push this branch to the etmc repo please?