search

p0p4k / pflowtts_pytorch

Unofficial implementation of NVIDIA P-Flow TTS paper
https://neurips.cc/virtual/2023/poster/69899
MIT License
214 stars 30 forks source link

Error when finetuning multi speaker from pretrained model #25

Closed ken2190 closed 8 months ago

ken2190 commented 8 months ago

I'm trying to finetune from pretrained model pflow-2000.ckpt on custom multi speaker dataset in German, but i get following error

  File "/home/ubuntu/DATA/pflowtts_pytorch/pflowtts_env/lib/python3.10/site-packages/lightning/pytorch/loops/evaluation_loop.py", line 134, in run
    self._evaluation_step(batch, batch_idx, dataloader_idx, dataloader_iter)
  File "/home/ubuntu/DATA/pflowtts_pytorch/pflowtts_env/lib/python3.10/site-packages/lightning/pytorch/loops/evaluation_loop.py", line 391, in _evaluation_step
    output = call._call_strategy_hook(trainer, hook_name, *step_args)
  File "/home/ubuntu/DATA/pflowtts_pytorch/pflowtts_env/lib/python3.10/site-packages/lightning/pytorch/trainer/call.py", line 309, in _call_strategy_hook
    output = fn(*args, **kwargs)
  File "/home/ubuntu/DATA/pflowtts_pytorch/pflowtts_env/lib/python3.10/site-packages/lightning/pytorch/strategies/strategy.py", line 402, in validation_step
    return self._forward_redirection(self.model, self.lightning_module, "validation_step", *args, **kwargs)
  File "/home/ubuntu/DATA/pflowtts_pytorch/pflowtts_env/lib/python3.10/site-packages/lightning/pytorch/strategies/strategy.py", line 633, in __call__
    wrapper_output = wrapper_module(*args, **kwargs)
  File "/home/ubuntu/DATA/pflowtts_pytorch/pflowtts_env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/ubuntu/DATA/pflowtts_pytorch/pflowtts_env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/ubuntu/DATA/pflowtts_pytorch/pflowtts_env/lib/python3.10/site-packages/torch/nn/parallel/distributed.py", line 1523, in forward
    else self._run_ddp_forward(*inputs, **kwargs)
  File "/home/ubuntu/DATA/pflowtts_pytorch/pflowtts_env/lib/python3.10/site-packages/torch/nn/parallel/distributed.py", line 1359, in _run_ddp_forward
    return self.module(*inputs, **kwargs)  # type: ignore[index]
  File "/home/ubuntu/DATA/pflowtts_pytorch/pflowtts_env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/ubuntu/DATA/pflowtts_pytorch/pflowtts_env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/ubuntu/DATA/pflowtts_pytorch/pflowtts_env/lib/python3.10/site-packages/lightning/pytorch/strategies/strategy.py", line 626, in wrapped_forward
    out = method(*_args, **_kwargs)
  File "/home/ubuntu/DATA/pflowtts_pytorch/pflow/models/baselightningmodule.py", line 150, in validation_step
    loss_dict, attn_dict = self.get_losses(batch)
  File "/home/ubuntu/DATA/pflowtts_pytorch/pflow/models/baselightningmodule.py", line 73, in get_losses
    dur_loss, prior_loss, diff_loss, attn = self(
  File "/home/ubuntu/DATA/pflowtts_pytorch/pflowtts_env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/ubuntu/DATA/pflowtts_pytorch/pflowtts_env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/ubuntu/DATA/pflowtts_pytorch/pflow/models/pflow_tts.py", line 150, in forward
    from pflow.utils.monotonic_align import maximum_path
  File "/home/ubuntu/DATA/pflowtts_pytorch/pflow/utils/monotonic_align/__init__.py", line 3, in <module>
    from pflow.utils.monotonic_align.core import maximum_path_c
ModuleNotFoundError: No module named 'pflow.utils.monotonic_align.core'

i also installed Cython library

(/home/ubuntu/DATA/pflowtts_pytorch/pflowtts_env) ubuntu@t4:~/DATA/pflowtts_pytorch$ pip install Cython                        
Requirement already satisfied: Cython in ./pflowtts_env/lib/python3.10/site-packages (3.0.8)

Could you share your multi speaker training config?

p0p4k commented 8 months ago

Did you follow instructions on readme? Build the monotonic alignment?