search

yangdongchao / UniAudio

The Open Source Code of UniAudio
http://dongchaoyang.top/UniAudio_demo/
475 stars 31 forks source link

Training Code availability #1

Open rishikksh20 opened 9 months ago

rishikksh20 commented 9 months ago

Hi @yangdongchao, I have checked your research paper and it's quite interesting to see that one model to do all. Are you planning to release training code , or just inference code?

Apart from that have you tried to include Speech to Speech Translation in this model, I think that is the only Speech generation task that is not included in the model.

Thanks

yangdongchao commented 9 months ago

Hi @yangdongchao, I have checked your research paper and it's quite interesting to see that one model to do all. Are you planning to release training code , or just inference code?

Apart from that have you tried to include Speech to Speech Translation in this model, I think that is the only Speech generation task that is not included in the model.

Thanks

Hi, rishikksh20 The whole training code will be released in this week, I am trying to write some documents to better using UniAudio. The speech to speech translation task also can be done with our framework. If you are interested in this ,we expect you can join us, and add the speech to speech translation task into UniAudio.

rishikksh20 commented 9 months ago

Currently facing this issue :

run.sh: line 37: utils/parse_options.sh: No such file or directory
Prepare LibriTTS dataset
split the data for 1 GPUs
cat: data/val/wav.scp: No such file or directory
run.sh: line 85: utils/split_scp.pl: No such file or directory
python3: can't open file '/content/UniAudio/UniAudio/egs/TTS/data_scripts/filter_scp.py': [Errno 2] No such file or directory
run.sh: line 85: utils/split_scp.pl: No such file or directory
python3: can't open file '/content/UniAudio/UniAudio/egs/TTS/data_scripts/filter_scp.py': [Errno 2] No such file or directory
Prepare text and audio sequence
prepare val ... 
run.sh: line 108: utils/run.pl: No such file or directory

Also requirement.txt I think not well formatted.

yangdongchao commented 9 months ago

Currently facing this issue :

run.sh: line 37: utils/parse_options.sh: No such file or directory
Prepare LibriTTS dataset
split the data for 1 GPUs
cat: data/val/wav.scp: No such file or directory
run.sh: line 85: utils/split_scp.pl: No such file or directory
python3: can't open file '/content/UniAudio/UniAudio/egs/TTS/data_scripts/filter_scp.py': [Errno 2] No such file or directory
run.sh: line 85: utils/split_scp.pl: No such file or directory
python3: can't open file '/content/UniAudio/UniAudio/egs/TTS/data_scripts/filter_scp.py': [Errno 2] No such file or directory
Prepare text and audio sequence
prepare val ... 
run.sh: line 108: utils/run.pl: No such file or directory

Also requirement.txt I think not well formatted.

please try to copy the utils folder from tools/kaldi/utils. It seems that the ln -s ../tools/kaldi/utils ./ command is not working in the run.sh. Please check it.

yangdongchao commented 9 months ago

Currently facing this issue :

run.sh: line 37: utils/parse_options.sh: No such file or directory
Prepare LibriTTS dataset
split the data for 1 GPUs
cat: data/val/wav.scp: No such file or directory
run.sh: line 85: utils/split_scp.pl: No such file or directory
python3: can't open file '/content/UniAudio/UniAudio/egs/TTS/data_scripts/filter_scp.py': [Errno 2] No such file or directory
run.sh: line 85: utils/split_scp.pl: No such file or directory
python3: can't open file '/content/UniAudio/UniAudio/egs/TTS/data_scripts/filter_scp.py': [Errno 2] No such file or directory
Prepare text and audio sequence
prepare val ... 
run.sh: line 108: utils/run.pl: No such file or directory

Also requirement.txt I think not well formatted.

please try to "bash requirements.sh" now.

rishikksh20 commented 9 months ago

Another issue :

Traceback (most recent call last):
  File "/content/UniAudio/UniAudio/egs/TTS/../../infer.py", line 17, in <module>
    from utils.dataloader import get_data_iterator_tokenizer_vocabulary
  File "/content/UniAudio/UniAudio/utils/dataloader.py", line 26, in <module>
    from tools.tokenizer.AudioTagging.audio_tagging_tokenizer import AudioTaggingTokenizer
ModuleNotFoundError: No module named 'tools.tokenizer.AudioTagging'
rishikksh20 commented 9 months ago

Couples of more minor bugs :

  1. String quotes are inconsistent
    https://github.com/yangdongchao/UniAudio/blob/0552aa3faa0314e87641f8cf4176975d95670814/UniAudio/tools/tokenizer/soundstream/AudioTokenizer.py#L43 
    self.ckpt_path = f'UniAudio/checkpoints/{tag}_model/model.pth'
  2. This line should be commented https://github.com/yangdongchao/UniAudio/blob/0552aa3faa0314e87641f8cf4176975d95670814/UniAudio/tools/tokenizer/soundstream/AudioTokenizer.py#L124
cantabile-kwok commented 9 months ago

Another issue :

Traceback (most recent call last):
  File "/content/UniAudio/UniAudio/egs/TTS/../../infer.py", line 17, in <module>
    from utils.dataloader import get_data_iterator_tokenizer_vocabulary
  File "/content/UniAudio/UniAudio/utils/dataloader.py", line 26, in <module>
    from tools.tokenizer.AudioTagging.audio_tagging_tokenizer import AudioTaggingTokenizer
ModuleNotFoundError: No module named 'tools.tokenizer.AudioTagging'

I think there are several other imports that don't even exist. Maybe this is distilled from a large project. I managed to fix this by moving those imports in the "if" branch of corresponding tokenizers.

GrahLnn commented 9 months ago

Same module error.