Open talka1 opened 2 years ago
I cant start the training at stage 3 anymore:
# python -m crank.bin.train --flag train --n_jobs 10 --conf conf/mlfb_vqvae.yml --checkpoint None --scpdir data/scp --featdir data/feature --expdir exp # Started at Wed Dec 22 19:20:31 UTC 2021 # 2021-12-22 19:20:40,711 (train:164) INFO: feature: {'label': 'mlfb', 'fs': 22050, 'fftl': 1024, 'win_length': 1024, 'hop_size': 128, 'fmin': 80, 'fmax': 7600, 'mlfb_dim': 80, 'n_iteration': 100, 'framems': 20, 'shiftms': 5.80499, 'mcep_dim': 34, 'mcep_alpha': 0.466, 'window_types': ['hann']} 2021-12-22 19:20:40,713 (train:164) INFO: input_feat_type: mlfb 2021-12-22 19:20:40,713 (train:164) INFO: output_feat_type: mlfb 2021-12-22 19:20:40,713 (train:164) INFO: trainer_type: vqvae 2021-12-22 19:20:40,714 (train:164) INFO: input_size: 80 2021-12-22 19:20:40,714 (train:164) INFO: output_size: 80 2021-12-22 19:20:40,714 (train:164) INFO: n_steps: 200000 2021-12-22 19:20:40,714 (train:164) INFO: dev_steps: 2000 2021-12-22 19:20:40,715 (train:164) INFO: n_steps_save_model: 350 2021-12-22 19:20:40,715 (train:164) INFO: n_steps_print_loss: 50 2021-12-22 19:20:40,715 (train:164) INFO: batch_size: 50 2021-12-22 19:20:40,715 (train:164) INFO: batch_len: 500 2021-12-22 19:20:40,715 (train:164) INFO: cache_dataset: True 2021-12-22 19:20:40,716 (train:164) INFO: spec_augment: False 2021-12-22 19:20:40,716 (train:164) INFO: n_spec_augment: 0 2021-12-22 19:20:40,716 (train:164) INFO: use_mcep_0th: False 2021-12-22 19:20:40,716 (train:164) INFO: ignore_scaler: [] 2021-12-22 19:20:40,716 (train:164) INFO: alpha: {'l1': 2, 'mse': 0, 'stft': 1, 'commit': 0.25, 'dict': 0.5, 'cycle': 0.1, 'ce': 1, 'adv': 1, 'real': 0.5, 'fake': 0.5, 'acgan': 1} 2021-12-22 19:20:40,717 (train:164) INFO: stft_params: {'fft_sizes': [64, 128], 'win_sizes': [64, 128], 'hop_sizes': [16, 32], 'logratio': 0} 2021-12-22 19:20:40,717 (train:164) INFO: optim: {'G': {'type': 'adam', 'lr': 0.0002, 'decay_size': 0.5, 'decay_step_size': 200000, 'clip_grad_norm': 0.0}, 'D': {'type': 'adam', 'lr': 5e-05, 'decay_size': 0.5, 'decay_step_size': 200000, 'clip_grad_norm': 0.0}, 'C': {'type': 'adam', 'lr': 0.0001, 'decay_size': 0.5, 'decay_step_size': 200000, 'clip_grad_norm': 0.0}, 'SPKRADV': {'type': 'adam', 'lr': 0.0001, 'decay_size': 0.5, 'decay_step_size': 200000, 'clip_grad_norm': 0.0}} 2021-12-22 19:20:40,717 (train:164) INFO: encoder_f0: False 2021-12-22 19:20:40,717 (train:164) INFO: decoder_f0: True 2021-12-22 19:20:40,718 (train:164) INFO: encoder_energy: False 2021-12-22 19:20:40,718 (train:164) INFO: decoder_energy: False 2021-12-22 19:20:40,718 (train:164) INFO: causal: False 2021-12-22 19:20:40,718 (train:164) INFO: causal_size: 0 2021-12-22 19:20:40,719 (train:164) INFO: use_spkr_embedding: True 2021-12-22 19:20:40,719 (train:164) INFO: spkr_embedding_size: 32 2021-12-22 19:20:40,719 (train:164) INFO: ema_flag: True 2021-12-22 19:20:40,719 (train:164) INFO: n_vq_stacks: 2 2021-12-22 19:20:40,719 (train:164) INFO: n_layers_stacks: [4, 3, 2] 2021-12-22 19:20:40,720 (train:164) INFO: n_layers: [2, 2, 2] 2021-12-22 19:20:40,720 (train:164) INFO: kernel_size: [5, 3, 3] 2021-12-22 19:20:40,720 (train:164) INFO: emb_dim: [64, 64, 64] 2021-12-22 19:20:40,720 (train:164) INFO: emb_size: [512, 512, 512] 2021-12-22 19:20:40,720 (train:164) INFO: use_spkradv_training: True 2021-12-22 19:20:40,721 (train:164) INFO: n_spkradv_layers: 3 2021-12-22 19:20:40,721 (train:164) INFO: spkradv_kernel_size: 3 2021-12-22 19:20:40,721 (train:164) INFO: spkradv_lambda: 0.1 2021-12-22 19:20:40,721 (train:164) INFO: use_spkr_classifier: True 2021-12-22 19:20:40,722 (train:164) INFO: n_spkr_classifier_layers: 8 2021-12-22 19:20:40,722 (train:164) INFO: spkr_classifier_kernel_size: 5 2021-12-22 19:20:40,722 (train:164) INFO: use_cyclic_training: False 2021-12-22 19:20:40,722 (train:164) INFO: n_steps_cycle_start: 50000 2021-12-22 19:20:40,722 (train:164) INFO: n_cycles: 1 2021-12-22 19:20:40,723 (train:164) INFO: n_steps_gan_start: 100000 2021-12-22 19:20:40,723 (train:164) INFO: gan_type: lsgan 2021-12-22 19:20:40,723 (train:164) INFO: use_residual_network: True 2021-12-22 19:20:40,723 (train:164) INFO: n_discriminator_layers: 2 2021-12-22 19:20:40,724 (train:164) INFO: n_discriminator_stacks: 4 2021-12-22 19:20:40,724 (train:164) INFO: discriminator_kernel_size: 5 2021-12-22 19:20:40,724 (train:164) INFO: discriminator_dropout: 0.25 2021-12-22 19:20:40,724 (train:164) INFO: train_first: D 2021-12-22 19:20:40,724 (train:164) INFO: switch_update: False 2021-12-22 19:20:40,725 (train:164) INFO: cvadv_flag: False 2021-12-22 19:20:40,725 (train:164) INFO: acgan_flag: False 2021-12-22 19:20:40,725 (train:164) INFO: encoder_detach: False 2021-12-22 19:20:40,726 (train:164) INFO: use_real_only_acgan: False 2021-12-22 19:20:40,726 (train:164) INFO: use_D_uv: True 2021-12-22 19:20:40,726 (train:164) INFO: use_D_spkrcode: True 2021-12-22 19:20:40,726 (train:164) INFO: use_vqvae_loss: True 2021-12-22 19:20:40,726 (train:164) INFO: n_steps_stop_generator: 0 Traceback (most recent call last): File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "/usr/lib/python3.7/runpy.py", line 85, in _run_code exec(code, run_globals) File "/content/drive/MyDrive/crank/crank/bin/train.py", line 231, in <module> main() File "/content/drive/MyDrive/crank/crank/bin/train.py", line 182, in main model = get_model(conf, spkr_size, device, scaler=scaler) File "/content/drive/MyDrive/crank/crank/bin/train.py", line 57, in get_model models = {"G": VQVAE2(conf, spkr_size=spkr_size, scaler=scaler).to(device)} File "/content/drive/MyDrive/crank/crank/net/module/vqvae2.py", line 52, in __init__ if self.conf["use_raw"]: KeyError: 'use_raw' # Accounting: time=12 threads=1 # Ended (code 1) at Wed Dec 22 19:20:43 UTC 2021, elapsed time 12 seconds
I cant start the training at stage 3 anymore: