The initialization consists of seven parts: (1) Get the dataset and data loader; (2) Get the tts model; (3) Dispatch the model from cpu to gpu; (4) Get the optimizer; (5) Load the checkpoint; (6) Get the scheduler; (7) Get the tensorboard summary.
Set the model device via local_rank instead of rank;