search

myshell-ai / MeloTTS

High-quality multi-lingual text-to-speech library by MyShell.ai. Support English, Spanish, French, Chinese, Japanese and Korean.
MIT License
4.84k stars 631 forks source link

Unable to train voice #170

Open dafogary opened 3 months ago

dafogary commented 3 months ago

Hi All

I have tried to ask this on Discord, a new user on that place, but it wouldn't let me, so thought I'd try here.

I have MeloTTS installed in a docker container, but I am running into some issues training a new voice. The preprocess_text.py appears to go through, but when I run:

bash train.sh data/config.json 1

I keep getting the error:

INFO:data:Loading the training dataset 2024-08-01 18:10:20.762 | INFO | data_utils:_filter:58 - Init dataset... 0it [00:00, ?it/s] 2024-08-01 18:10:20.763 | ERROR | data_utils:_filter:78 - No valid data loaded. Please check your training files. ERROR:data:Training dataset is empty. Please check the data paths and files.

In my train.list, I have:

/app/dafo_files/DaveSampleVoiceshort.wav|Dave|EN|The script from the user.

In my data directory, I have:

root@cc432df4ef3d:/app/melo/data# ls -l total 20 -rw-r--r-- 1 root root 155 Aug 1 17:54 DAFOTrain.list -rw-r--r-- 1 root root 542 Aug 1 17:55 DAFOTrain.list.cleaned -rw-r--r-- 1 root root 3998 Aug 1 17:55 config.json drwxrwxr-x 2 root root 4096 Jul 25 09:57 example -rw-r--r-- 1 root root 0 Aug 1 17:55 train.list -rw-r--r-- 1 root root 542 Aug 1 17:55 val.list

Anyone have any suggestions what I'm doing wrong?

I have cloned the repository, the original just wasn't working for me, I can get much further with my repository. You can see what I have done at https://github.com/dafogary/MeloTTS

Zengyi-Qin commented 3 months ago

It seems to be the dataset directory problem. Could you dive into the code and see where the code is searching for the dataset?

dafogary commented 2 months ago

Apologies for the delay, it's been hectic in recent weeks. Thought I need to have a good sit down and go throuigh this.

Inititially, it looked like I hadn't got the NVIDIA aligned with my docker container, which I now have. But I am still getting the error when I run:

root@local:/app/melo# python3 preprocess_text.py --metadata data/DAFOTrain.list

I get

 0%|                                                                                                                                                                                 | 0/1 [00:00<?, ?it/s]/usr/local/lib/python3.9/site-packages/transformers/modeling_utils.py:415: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  return torch.load(checkpoint_file, map_location="cpu")
Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['cls.seq_relationship.weight', 'cls.seq_relationship.bias']
- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).

Then when I run:

bash train.sh data/config.json 1

I get:

2024-08-26 16:16:27.606 | INFO     | data_utils:_filter:58 - Init dataset...
0it [00:00, ?it/s]
2024-08-26 16:16:27.607 | ERROR    | data_utils:_filter:78 - No valid data loaded. Please check your training files.
ERROR:data:Training dataset is empty. Please check the data paths and files.

In my data directory I have:

-rw-r--r-- 1 root root  149 Aug 26 16:11 DAFOTrain.list
-rw-r--r-- 1 root root  536 Aug 26 16:12 DAFOTrain.list.cleaned
-rw-r--r-- 1 root root 3998 Aug 26 16:12 config.json
drwxrwxr-x 2 root root 4096 Jul 31 09:57 example
-rw-r--r-- 1 root root    0 Aug 26 16:12 train.list
-rw-r--r-- 1 root root  536 Aug 26 16:12 val.list
drwxr-xr-x 2 root root 4096 Aug 26 16:02 wavs

Where are the datasets locations set?

Running:

grep -r "dataset" /home/user/Documents/GitHub/MeloTTS

I just get:

/home/user/Documents/GitHub/MeloTTS/melo/train.py:        logger.info("Loading the training dataset")
/home/user/Documents/GitHub/MeloTTS/melo/train.py:        train_dataset = TextAudioSpeakerLoader(hps.data.training_files, hps.data)
/home/user/Documents/GitHub/MeloTTS/melo/train.py:        if not train_dataset.lengths:
/home/user/Documents/GitHub/MeloTTS/melo/train.py:            logger.error("Training dataset is empty. Please check the data paths and files.")
/home/user/Documents/GitHub/MeloTTS/melo/train.py:            train_dataset,
/home/user/Documents/GitHub/MeloTTS/melo/train.py:            train_dataset,
/home/user/Documents/GitHub/MeloTTS/melo/train.py:            eval_dataset = TextAudioSpeakerLoader(hps.data.validation_files, hps.data)
/home/user/Documents/GitHub/MeloTTS/melo/train.py:                eval_dataset,
/home/user/Documents/GitHub/MeloTTS/melo/data_utils.py:        logger.info("Init dataset...")
/home/user/Documents/GitHub/MeloTTS/melo/data_utils.py:        dataset,
/home/user/Documents/GitHub/MeloTTS/melo/data_utils.py:        super().__init__(dataset, num_replicas=num_replicas, rank=rank, shuffle=shuffle)
/home/user/Documents/GitHub/MeloTTS/melo/data_utils.py:        self.lengths = dataset.lengths

When I run the command:

python3 preprocess_text.py --metadata data/DAFOTrain.list

The train list is in the data directory, with the audio in the data/wavs directory. But running:

bash train.sh data/config.json 1

It isn't happy.

When I run the preprocess_text.py file I do get:

/usr/local/lib/python3.9/site-packages/transformers/modeling_utils.py:415: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  return torch.load(checkpoint_file, map_location="cpu")
Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['cls.seq_relationship.bias', 'cls.seq_relationship.weight']
- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).

Could that be the issue?

Any help is appreciated

FXzeng6 commented 1 month ago

很抱歉耽搁了，最近几周很忙。我想我需要好好坐下来，然后完成这个。

最初，我似乎没有将 NVIDIA 与我的 docker 容器对齐，而我现在拥有了这个容器。但是我在运行时仍然收到错误：

root@local:/app/melo# python3 preprocess_text.py --metadata data/DAFOTrain.list

我得到

0%|                                                                                                                                                                                 | 0/1 [00:00<?, ?it/s]/usr/local/lib/python3.9/site-packages/transformers/modeling_utils.py:415: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
 return torch.load(checkpoint_file, map_location="cpu")
Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['cls.seq_relationship.weight', 'cls.seq_relationship.bias']
- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).

然后当我运行时：

bash train.sh data/config.json 1

我得到：

2024-08-26 16:16:27.606 | INFO     | data_utils:_filter:58 - Init dataset...
0it [00:00, ?it/s]
2024-08-26 16:16:27.607 | ERROR    | data_utils:_filter:78 - No valid data loaded. Please check your training files.
ERROR:data:Training dataset is empty. Please check the data paths and files.

在我的数据目录中，我有：

-rw-r--r-- 1 root root  149 Aug 26 16:11 DAFOTrain.list
-rw-r--r-- 1 root root  536 Aug 26 16:12 DAFOTrain.list.cleaned
-rw-r--r-- 1 root root 3998 Aug 26 16:12 config.json
drwxrwxr-x 2 root root 4096 Jul 31 09:57 example
-rw-r--r-- 1 root root    0 Aug 26 16:12 train.list
-rw-r--r-- 1 root root  536 Aug 26 16:12 val.list
drwxr-xr-x 2 root root 4096 Aug 26 16:02 wavs

数据集位置设置在哪里？

运行：

grep -r "dataset" /home/user/Documents/GitHub/MeloTTS

我只是得到：

/home/user/Documents/GitHub/MeloTTS/melo/train.py:        logger.info("Loading the training dataset")
/home/user/Documents/GitHub/MeloTTS/melo/train.py:        train_dataset = TextAudioSpeakerLoader(hps.data.training_files, hps.data)
/home/user/Documents/GitHub/MeloTTS/melo/train.py:        if not train_dataset.lengths:
/home/user/Documents/GitHub/MeloTTS/melo/train.py:            logger.error("Training dataset is empty. Please check the data paths and files.")
/home/user/Documents/GitHub/MeloTTS/melo/train.py:            train_dataset,
/home/user/Documents/GitHub/MeloTTS/melo/train.py:            train_dataset,
/home/user/Documents/GitHub/MeloTTS/melo/train.py:            eval_dataset = TextAudioSpeakerLoader(hps.data.validation_files, hps.data)
/home/user/Documents/GitHub/MeloTTS/melo/train.py:                eval_dataset,
/home/user/Documents/GitHub/MeloTTS/melo/data_utils.py:        logger.info("Init dataset...")
/home/user/Documents/GitHub/MeloTTS/melo/data_utils.py:        dataset,
/home/user/Documents/GitHub/MeloTTS/melo/data_utils.py:        super().__init__(dataset, num_replicas=num_replicas, rank=rank, shuffle=shuffle)
/home/user/Documents/GitHub/MeloTTS/melo/data_utils.py:        self.lengths = dataset.lengths

当我运行命令时：

python3 preprocess_text.py --metadata data/DAFOTrain.list

train list 在 data 目录中，audio 在 data/wavs 目录中。但是运行：

bash train.sh data/config.json 1

它不快乐。

当我运行 preprocess_text.py 文件时，我得到：

/usr/local/lib/python3.9/site-packages/transformers/modeling_utils.py:415: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  return torch.load(checkpoint_file, map_location="cpu")
Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['cls.seq_relationship.bias', 'cls.seq_relationship.weight']
- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).

这可能是问题所在吗？

任何帮助都值得赞赏

You can't use /app/dafo_files/DaveSampleVoiceshort.wav as the path, you should use /app/melo/data/wavs/DaveSampleVoiceshort.wav, your DAFOTrain.list may be processed incorrectly