[QUESTION] How to time the code

[Weifan1226]

Hi Megatron team!

I wanted to time some of Megatron's code blocks, but didn't find a good way to do it. But I was surprised to find code like this:

          if config.timers is not None:
                  config.timers('layernorm-grads-all-reduce', log_level=1).start(
                      barrier=config.barrier_with_L1_time
                  )
              _allreduce_layernorm_grads(model, config)
              if config.timers is not None:
                  config.timers('layernorm-grads-all-reduce').stop()

Can I us config.timers() to time the code? If yes, how can i see the output from config.timers.

Thanks!