jaywalnut310 / vits

VITS: Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech
https://jaywalnut310.github.io/vits-demo/index.html
MIT License
6.53k stars 1.22k forks source link

Fail to run train.py #68

Closed l1006986533 closed 1 year ago

l1006986533 commented 1 year ago

I tried to run train.py but failed. Here is the result.

(vits) root@237a4940d107:/home/vits# python train.py INFO:dsq:{'train': {'log_interval': 50, 'eval_interval': 500, 'seed': 1234, 'epochs': 20000, 'learning_rate': 0.0002, 'betas': [0.8, 0.99], 'eps': 1e-09, 'batch_size': 32, 'fp16_run': True, 'lr_decay': 0.999875, 'segment_size': 8192, 'init_lr_ratio': 1, 'warmup_epochs': 0, 'c_mel': 45, 'c_kl': 1.0}, 'data': {'training_files': 'filelists/dsq.txt.cleaned', 'validation_files': 'filelists/dsq_val.txt.cleaned', 'text_cleaners': ['chinese_cleaners'], 'max_wav_value': 32768.0, 'sampling_rate': 22050, 'filter_length': 1024, 'hop_length': 256, 'win_length': 1024, 'n_mel_channels': 80, 'mel_fmin': 0.0, 'mel_fmax': None, 'add_blank': True, 'n_speakers': 0, 'cleaned_text': True}, 'model': {'inter_channels': 192, 'hidden_channels': 192, 'filter_channels': 768, 'n_heads': 2, 'n_layers': 6, 'kernel_size': 3, 'p_dropout': 0.1, 'resblock': '1', 'resblock_kernel_sizes': [3, 7, 11], 'resblock_dilation_sizes': [[1, 3, 5], [1, 3, 5], [1, 3, 5]], 'upsample_rates': [8, 8, 2, 2], 'upsample_initial_channel': 512, 'upsample_kernel_sizes': [16, 16, 4, 4], 'n_layers_q': 3, 'use_spectral_norm': False}, 'model_dir': './logs/dsq'} Exception ignored in: <bound method _MultiProcessingDataLoaderIter.del of <torch.utils.data.dataloader._MultiProcessingDataLoaderIter object at 0x7f9317d5fcc0>> Traceback (most recent call last): File "/root/miniconda3/envs/vits/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 1101, in del Exception ignored in: <bound method _MultiProcessingDataLoaderIter.del of <torch.utils.data.dataloader._MultiProcessingDataLoaderIter object at 0x7f23df7f6588>> Traceback (most recent call last): File "/root/miniconda3/envs/vits/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 1101, in del self._shutdown_workers() File "/root/miniconda3/envs/vits/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 1050, in _shutdown_workers self._shutdown_workers() File "/root/miniconda3/envs/vits/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 1050, in _shutdown_workers if not self._shutdown: AttributeError: '_MultiProcessingDataLoaderIter' object has no attribute '_shutdown' if not self._shutdown: AttributeError: '_MultiProcessingDataLoaderIter' object has no attribute '_shutdown' Traceback (most recent call last): File "train.py", line 290, in main() File "train.py", line 50, in main mp.spawn(run, nprocs=n_gpus, args=(n_gpus, hps,)) File "/root/miniconda3/envs/vits/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 200, in spawn return start_processes(fn, args, nprocs, join, daemon, start_method='spawn') File "/root/miniconda3/envs/vits/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 158, in start_processes while not context.join(): File "/root/miniconda3/envs/vits/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 119, in join raise Exception(msg) Exception:

-- Process 1 terminated with the following error: Traceback (most recent call last): File "/root/miniconda3/envs/vits/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 20, in _wrap fn(i, args) File "/home/vits/train.py", line 119, in run train_and_evaluate(rank, epoch, hps, [net_g, net_d], [optim_g, optim_d], [scheduler_g, scheduler_d], scaler, [train_loader, None], None, None) File "/home/vits/train.py", line 137, in train_and_evaluate for batch_idx, (x, x_lengths, spec, spec_lengths, y, y_lengths) in enumerate(train_loader): File "/root/miniconda3/envs/vits/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 291, in iter return _MultiProcessingDataLoaderIter(self) File "/root/miniconda3/envs/vits/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 691, in init super(_MultiProcessingDataLoaderIter, self).init(loader) File "/root/miniconda3/envs/vits/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 349, in init self._sampler_iter = iter(self._index_sampler) File "/home/vits/data_utils.py", line 358, in iter ids_bucket = ids_bucket + ids_bucket (rem // len_bucket) + ids_bucket[:(rem % len_bucket)] ZeroDivisionError: integer division or modulo by zero

I have checked everything but found nothing incorrect. Still the train.py fail to run. Can someone help me?

l1006986533 commented 1 year ago

The dubugger shows that len_bucket=0

l1006986533 commented 1 year ago

fix by #54

code in lines 323 in data_utils.py:

        for i in range(len(buckets) - 2, -1, -1):
            if len(buckets[i]) == 0:
                buckets.pop(i)
                self.boundaries.pop(i+1)