search

insunhwang89 / StyleVC

MIT License
29 stars 3 forks source link

Multi gpu training not working, hangs on iter 0 #9

Open skol101 opened 1 year ago

skol101 commented 1 year ago

Log

Iter 0      total 101.888 mel 47.537 mel_post 48.427 pitch 2.577  style 0.500  adv_spk 2.846  adv_spk 0.716 %
Saving model and optimizer state at iteration 0 to outputs/vctk_142/checkpoint_G_0
Saving model and optimizer state at iteration 0 to outputs/vctk_142/checkpoint_D_0
gpu id ---------------->  0,2
seen speakers! 134 unseen speakers! 8
^CTraceback (most recent call last):
  File "trainer.py", line 132, in <module>
    mp.spawn(main, nprocs=num_gpus, args=(num_gpus, batch_size,))
  File "/home/sk/anaconda3/envs/vc/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 230, in spawn
    return start_processes(fn, args, nprocs, join, daemon, start_method='spawn')
  File "/home/sk/anaconda3/envs/vc/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 188, in start_processes
    while not context.join():
  File "/home/sk/anaconda3/envs/vc/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 99, in join
    ready = multiprocessing.connection.wait(
  File "/home/sk/anaconda3/envs/vc/lib/python3.8/multiprocessing/connection.py", line 931, in wait
    ready = selector.select(timeout)
  File "/home/sk/anaconda3/envs/vc/lib/python3.8/selectors.py", line 415, in select
    fd_event_list = self._selector.poll(timeout)
insunhwang89 commented 1 year ago

We don't support Multi GPU, we implemented it for testing and haven't confirmed it.