training 8 speaker 2 lingual (en, pl) model kali linux wsl2

[gregniuki]

my cashe size is 60 gb and jsonl 64 mb, 1 epoch 2000 iterations with batch size 32 takes 50 minutes on rtx 4090 using 33 gb vram im training useing wsl2 kali linux Is something bottlenecking gpu only draws 100 wat ? Training is working i dont use max-phoneme-ids 400 because some of sentences are longer like 600

└─$ python3 -m piper_train.preprocess --language pl --input-dir ~/skocz/ --output-dir ~/tskocz/ --dataset-format ljspeech --sample-rate 22050 INFO:preprocess:8 speakers detected INFO:preprocess:Wrote dataset config INFO:preprocess:Processing 29745 utterance(s) with 20 worker(s)

┌──(.venv)(gregniuki㉿gregsacer)-[~/piper/src/python] └─$ python3 -m piper_train --dataset-dir ~/tskocz/ --accelerator 'gpu' --devices 1 --batch-size 32 --validation-split 0.01 --num-test-examples 4 --max_epochs 10000 --resume_from_single_speaker_checkpoint ~/high/epoch=2674-step=961542.ckpt --checkpoint-epochs 4 --precision 32 --quality high DEBUG:piper_train:Namespace(dataset_dir='/home/gregniuki/tskocz/', checkpoint_epochs=4, quality='high', resume_from_single_speaker_checkpoint='/home/gregniuki/high/epoch=2674-step=961542.ckpt', logger=True, enable_checkpointing=True, default_root_dir=None, gradient_clip_val=None, gradient_clip_algorithm=None, num_nodes=1, num_processes=None, devices='1', gpus=None, auto_select_gpus=False, tpu_cores=None, ipus=None, enable_progress_bar=True, overfit_batches=0.0, track_grad_norm=-1, check_val_every_n_epoch=1, fast_dev_run=False, accumulate_grad_batches=None, max_epochs=10000, min_epochs=None, max_steps=-1, min_steps=None, max_time=None, limit_train_batches=None, limit_val_batches=None, limit_test_batches=None, limit_predict_batches=None, val_check_interval=None, log_every_n_steps=50, accelerator='gpu', strategy=None, sync_batchnorm=False, precision=32, enable_model_summary=True, weights_save_path=None, num_sanity_val_steps=2, resume_from_checkpoint=None, profiler=None, benchmark=None, deterministic=None, reload_dataloaders_every_n_epochs=0, auto_lr_find=False, replace_sampler_ddp=True, detect_anomaly=False, auto_scale_batch_size=False, plugins=None, amp_backend='native', amp_level=None, move_metrics_to_cpu=False, multiple_trainloader_mode='max_size_cycle', batch_size=32, validation_split=0.01, num_test_examples=4, max_phoneme_ids=None, hidden_channels=192, inter_channels=192, filter_channels=768, n_layers=6, n_heads=2, seed=1234) GPU available: True (cuda), used: True TPU available: False, using: 0 TPU cores IPU available: False, using: 0 IPUs HPU available: False, using: 0 HPUs DEBUG:piper_train:Checkpoints will be saved every 4 epoch(s) DEBUG:vits.dataset:Loading dataset: /home/gregniuki/tskocz/dataset.jsonl DEBUG:piper_train:Resuming from single-speaker checkpoint: /home/gregniuki/high/epoch=2674-step=961542.ckpt DEBUG:fsspec.local:open file: /home/gregniuki/high/epoch=2674-step=961542.ckpt DEBUG:vits.lightning:No dataset to load DEBUG:piper_train:dec.cond.weight is not in the checkpoint DEBUG:piper_train:dec.cond.bias is not in the checkpoint DEBUG:piper_train:enc_q.enc.cond_layer.bias is not in the checkpoint DEBUG:piper_train:enc_q.enc.cond_layer.weight_g is not in the checkpoint DEBUG:piper_train:enc_q.enc.cond_layer.weight_v is not in the checkpoint DEBUG:piper_train:flow.flows.0.enc.cond_layer.bias is not in the checkpoint DEBUG:piper_train:flow.flows.0.enc.cond_layer.weight_g is not in the checkpoint DEBUG:piper_train:flow.flows.0.enc.cond_layer.weight_v is not in the checkpoint DEBUG:piper_train:flow.flows.2.enc.cond_layer.bias is not in the checkpoint DEBUG:piper_train:flow.flows.2.enc.cond_layer.weight_g is not in the checkpoint DEBUG:piper_train:flow.flows.2.enc.cond_layer.weight_v is not in the checkpoint DEBUG:piper_train:flow.flows.4.enc.cond_layer.bias is not in the checkpoint DEBUG:piper_train:flow.flows.4.enc.cond_layer.weight_g is not in the checkpoint DEBUG:piper_train:flow.flows.4.enc.cond_layer.weight_v is not in the checkpoint DEBUG:piper_train:flow.flows.6.enc.cond_layer.bias is not in the checkpoint DEBUG:piper_train:flow.flows.6.enc.cond_layer.weight_g is not in the checkpoint DEBUG:piper_train:flow.flows.6.enc.cond_layer.weight_v is not in the checkpoint DEBUG:piper_train:dp.cond.weight is not in the checkpoint DEBUG:piper_train:dp.cond.bias is not in the checkpoint DEBUG:piper_train:emb_g.weight is not in the checkpoint INFO:piper_train:Successfully converted single-speaker checkpoint to multi-speaker Missing logger folder: /home/gregniuki/tskocz/lightning_logs LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0] DEBUG:fsspec.local:open file: /home/gregniuki/tskocz/lightning_logs/version_0/hparams.yaml /home/gregniuki/piper/src/python/.venv/lib/python3.11/site-packages/pytorch_lightning/trainer/connectors/data_connector.py:236: PossibleUserWarning: The dataloader, val_dataloader 0, does not have many workers which may be a bottleneck. Consider increasing the value of the num_workers argument(try 20 which is the number of cpus on this machine) in theDataLoader` init to improve performance.

where i should change hparams num_workers 1 to 20 ? does kali linux wsl2 will allow it to increase gpu utilization?

[gregniuki]

?