It pauses and wont train

[Gleenan]

2023-10-20 14:52:11 | INFO | main | "runtime\python.exe" infer/modules/train/train.py -e "pog" -sr 48k -f0 1 -bs 4 -g 0 -te 250 -se 50 -pg assets/pretrained_v2/f0G48k.pth -pd assets/pretrained_v2/f0D48k.pth -l 0 -c 0 -sw 0 -v v2 INFO:pog:{'train': {'log_interval': 200, 'seed': 1234, 'epochs': 20000, 'learning_rate': 0.0001, 'betas': [0.8, 0.99], 'eps': 1e-09, 'batch_size': 4, 'fp16_run': False, 'lr_decay': 0.999875, 'segment_size': 12800, 'init_lr_ratio': 1, 'warmup_epochs': 0, 'c_mel': 45, 'c_kl': 1.0}, 'data': {'max_wav_value': 32768.0, 'sampling_rate': 40000, 'filter_length': 2048, 'hop_length': 400, 'win_length': 2048, 'n_mel_channels': 125, 'mel_fmin': 0.0, 'mel_fmax': None, 'training_files': './logs\pog/filelist.txt'}, 'model': {'inter_channels': 192, 'hidden_channels': 192, 'filter_channels': 768, 'n_heads': 2, 'n_layers': 6, 'kernel_size': 3, 'p_dropout': 0, 'resblock': '1', 'resblock_kernel_sizes': [3, 7, 11], 'resblock_dilation_sizes': [[1, 3, 5], [1, 3, 5], [1, 3, 5]], 'upsample_rates': [10, 10, 2, 2], 'upsample_initial_channel': 512, 'upsample_kernel_sizes': [16, 16, 4, 4], 'use_spectral_norm': False, 'gin_channels': 256, 'spk_embed_dim': 109}, 'model_dir': './logs\pog', 'experiment_dir': './logs\pog', 'save_every_epoch': 50, 'name': 'pog', 'total_epoch': 250, 'pretrainG': 'assets/pretrained_v2/f0G48k.pth', 'pretrainD': 'assets/pretrained_v2/f0D48k.pth', 'version': 'v2', 'gpus': '0', 'sample_rate': '48k', 'if_f0': 1, 'if_latest': 0, 'save_every_weights': '0', 'if_cache_data_in_gpu': 0} INFO:torch.distributed.distributed_c10d:Added key: store_based_barrier_key:1 to store for rank: 0 INFO:torch.distributed.distributed_c10d:Rank 0: Completed store-based barrier for key:store_based_barrier_key:1 with 1 nodes. DEBUG:infer.lib.infer_pack.models:gin_channels: 256, self.spk_embed_dim: 109 INFO:pog:loaded pretrained assets/pretrained_v2/f0G48k.pth Process Process-1: Traceback (most recent call last): File "D:\Games\RVC1006Nvidia\infer\modules\train\train.py", line 213, in run utils.latest_checkpoint_path(hps.modeldir, "D*.pth"), net_d, optim_d File "D:\Games\RVC1006Nvidia\infer\lib\train\utils.py", line 213, in latest_checkpoint_path x = f_list[-1] IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "multiprocessing\process.py", line 315, in _bootstrap File "multiprocessing\process.py", line 108, in run File "D:\Games\RVC1006Nvidia\infer\modules\train\train.py", line 233, in run net_g.module.load_state_dict( File "D:\Games\RVC1006Nvidia\runtime\lib\site-packages\torch\nn\modules\module.py", line 2041, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for SynthesizerTrnMs768NSFsid: size mismatch for dec.ups.0.weight_v: copying a param with shape torch.Size([512, 256, 24]) from checkpoint, the shape in current model is torch.Size([512, 256, 16]). size mismatch for dec.ups.1.weight_v: copying a param with shape torch.Size([256, 128, 20]) from checkpoint, the shape in current model is torch.Size([256, 128, 16]).

[liulfy]

same problem

[github-actions[bot]]

This issue was closed because it has been inactive for 15 days since being marked as stale.