horovod mnist.py has higher utilization number. what does it do?

[pint1022]

With the same model running with tfds, the utilization of gpu is about 20%. look into the execution sequence to capture detail GPU calls in horovod mninst.py test.