search

CSTR-Edinburgh / merlin

This is now the official location of the Merlin project.
http://www.cstr.ed.ac.uk/projects/merlin/
Apache License 2.0
1.31k stars 441 forks source link

Voice Conversion - Not converging after 50 epochs #344

Open vivek-sethia opened 6 years ago

vivek-sethia commented 6 years ago

How many epochs are required to get a decent voice conversion? I am using the default dataset of CMU Arctic and folllowed all the steps but the converted voice are not all audible.

felipeespic commented 6 years ago

Hi, which recipe are you using (s1 or s2)?

vivek-sethia commented 6 years ago

I am using s1 recipe

felipeespic commented 6 years ago

Didn't you try with s2?

vivek-sethia commented 6 years ago

Doess2 recipe works with python 2 only? Since I am facing a lot of error beacuse of python 3

felipeespic commented 6 years ago

Yes, I think so, Python 2.7

vivek-sethia commented 6 years ago

@felipeespic After creating new environment with python 2.7 and running s2 script, I got stuck in the second step

Here is the log 

Prepare acoustic features using MAGPHASE vocoder...

Analysing file: arctic_a0020.wav................................
Analysing file: arctic_a0001.wav................................

Analysing file: arctic_a0039.wav................................

Analysing file: arctic_a0058.wav................................
Extracting epochs with REAPER...
Extracting epochs with REAPER...
Extracting epochs with REAPER...
Extracting epochs with REAPER...
Residual symmetry: P:1057.966064  N:2080.696289  MEAN:-1.834260
Residual symmetry: P:779.040894  N:1617.814087  MEAN:-0.248217
Residual symmetry: P:565.581360  N:1208.755005  MEAN:0.239541
Residual symmetry: P:593.399414  N:1381.622070  MEAN:0.091667
/bin/sh: 1: /home/ubuntu/merlin/tools/magphase/tools/bin/mcep: not found
/bin/sh: 1: /home/ubuntu/merlin/tools/magphase/tools/bin/mcep: not found
/bin/sh: 1: /home/ubuntu/merlin/tools/magphase/tools/bin/mcep: not found
/bin/sh: 1: /home/ubuntu/merlin/tools/magphase/tools/bin/mcep: not found
Traceback (most recent call last):
  File "/home/ubuntu/merlin/misc/scripts/vocoder/magphase/extract_features_for_merlin.py", line 82, in <module>
    lu.run_multithreaded(feat_extraction, l_wavfiles, out_dir)
  File "/home/ubuntu/merlin/tools/magphase/src/libutils.py", line 62, in run_multithreaded
    results = pool.map(func_wrapper, l_iterable_args)
  File "/home/ubuntu/anaconda2/lib/python2.7/multiprocessing/pool.py", line 253, in map
    return self.map_async(func, iterable, chunksize).get()
  File "/home/ubuntu/anaconda2/lib/python2.7/multiprocessing/pool.py", line 572, in get
    raise self._value
ValueError: operands could not be broadcast together with shapes (0,10) (396,1) 
/bin/sh: 1: /home/ubuntu/merlin/tools/magphase/tools/bin/mcep: not found
/bin/sh: 1: /home/ubuntu/merlin/tools/magphase/tools/bin/mcep: not found
/bin/sh: 1: /home/ubuntu/merlin/tools/magphase/tools/bin/mcep: not found
/bin/sh: 1: /home/ubuntu/merlin/tools/magphase/tools/bin/mcep: not found
/bin/sh: 1: /home/ubuntu/merlin/tools/magphase/tools/bin/mcep: not found
/bin/sh: 1: /home/ubuntu/merlin/tools/magphase/tools/bin/mcep: not found
/bin/sh: 1: /home/ubuntu/merlin/tools/magphase/tools/bin/mcep: not found
/bin/sh: 1: /home/ubuntu/merlin/tools/magphase/tools/bin/mcep: not found
dreamk73 commented 6 years ago

mcep is an SPTK program so it looks like you have the wrong path in your conf file. Mine is located in ~/merlin/tools/bin/SPTK-3.9

On Wed, May 23, 2018 at 7:57 AM Vivek Sethia notifications@github.com wrote:

After creating new environment with python 2.7 and running s2 script, I got stuck in the second step

Here is the log

Prepare acoustic features using MAGPHASE vocoder...

Analysing file: arctic_a0020.wav................................ Analysing file: arctic_a0001.wav................................

Analysing file: arctic_a0039.wav................................

Analysing file: arctic_a0058.wav................................ Extracting epochs with REAPER... Extracting epochs with REAPER... Extracting epochs with REAPER... Extracting epochs with REAPER... Residual symmetry: P:1057.966064 N:2080.696289 MEAN:-1.834260 Residual symmetry: P:779.040894 N:1617.814087 MEAN:-0.248217 Residual symmetry: P:565.581360 N:1208.755005 MEAN:0.239541 Residual symmetry: P:593.399414 N:1381.622070 MEAN:0.091667 /bin/sh: 1: /home/ubuntu/merlin/tools/magphase/tools/bin/mcep: not found /bin/sh: 1: /home/ubuntu/merlin/tools/magphase/tools/bin/mcep: not found /bin/sh: 1: /home/ubuntu/merlin/tools/magphase/tools/bin/mcep: not found /bin/sh: 1: /home/ubuntu/merlin/tools/magphase/tools/bin/mcep: not found Traceback (most recent call last): File "/home/ubuntu/merlin/misc/scripts/vocoder/magphase/extract_features_for_merlin.py", line 82, in lu.run_multithreaded(feat_extraction, l_wavfiles, out_dir) File "/home/ubuntu/merlin/tools/magphase/src/libutils.py", line 62, in run_multithreaded results = pool.map(func_wrapper, l_iterable_args) File "/home/ubuntu/anaconda2/lib/python2.7/multiprocessing/pool.py", line 253, in map return self.map_async(func, iterable, chunksize).get() File "/home/ubuntu/anaconda2/lib/python2.7/multiprocessing/pool.py", line 572, in get raise self._value ValueError: operands could not be broadcast together with shapes (0,10) (396,1) /bin/sh: 1: /home/ubuntu/merlin/tools/magphase/tools/bin/mcep: not found /bin/sh: 1: /home/ubuntu/merlin/tools/magphase/tools/bin/mcep: not found /bin/sh: 1: /home/ubuntu/merlin/tools/magphase/tools/bin/mcep: not found /bin/sh: 1: /home/ubuntu/merlin/tools/magphase/tools/bin/mcep: not found /bin/sh: 1: /home/ubuntu/merlin/tools/magphase/tools/bin/mcep: not found /bin/sh: 1: /home/ubuntu/merlin/tools/magphase/tools/bin/mcep: not found /bin/sh: 1: /home/ubuntu/merlin/tools/magphase/tools/bin/mcep: not found /bin/sh: 1: /home/ubuntu/merlin/tools/magphase/tools/bin/mcep: not found

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/CSTR-Edinburgh/merlin/issues/344#issuecomment-391378687, or mute the thread https://github.com/notifications/unsubscribe-auth/ASbibN9PDZN3GFrpdhQvN6mLKwAADyHJks5t1XjXgaJpZM4TVC-v .

vivek-sethia commented 6 years ago

I checked my folder /merlin/tools/bin/SPTK-3.9, it is empty. Do you think I missed some step of installation?

vivek-sethia commented 6 years ago

Finally the scripts after installing csh which was the cause of my SPTK installation fail and hence other issues arose.

I was able to use s2 script for voice conversion. Thanks @felipeespic

felipeespic commented 6 years ago

You are welcome

vivek-sethia commented 6 years ago

@felipeespic When I am trying to run the script on some other dataset like https://smartlaboratory.org/ravdess/ . In the 2nd step while extracting acoustic features, I am getting an error like this.

Inverting signal
/home/ubuntu/merlin/tools/magphase/src/magphase.py:465: RuntimeWarning: invalid value encountered in divide

Can you know what needs to be done with the data? I have already converted sampling frequency of the wav files to 16000Hz

felipeespic commented 6 years ago

Could you send me a wav file that is producing the problem, so I can fix the code to deal with those cases, please?

Also, did the demo recipe work?

vivek-sethia commented 6 years ago

The demo recipe did work but it doesn't not work with other data I am using. You can find the wav files here

vivek-sethia commented 6 years ago

@felipeespic Also if I can find out some documentation or tutorials regarding this voice conversion method provided here, it would be nice to understand the overall process.

felipeespic commented 6 years ago

I realised that actually it was just a warning, so it shouldn't affect the performance. Anyway, I just got rid of that warning and I pushed it to the master, so you can download the new version of MagPhase if wanted.

For the documentation, unfortunately I don't have any by now, but if I find something I will post it.

Let me know how it goes. Thanks!

vivek-sethia commented 6 years ago

@felipeespic Do you think I can use this as recipe for emotonal voice conversion, i.e neutral to emotional speech?

If yes, can you give some pointers. I am reading this paper about https://www.researchgate.net/publication/306928922_Emotional_voice_conversion_using_deep_neural_networks_with_MCC_and_F0_features and was thinking if could use a similar approach with this tool.

vivek-sethia commented 6 years ago

@felipeespic Thank you for the recipe and supporting regularly with the issues.

Is it possible talk to you on Gitter for some time, so that I can clarify few of the things. It will be not long, I will just need to ask few things.

Since that will help me a bit to get a clear picture of the recipe and also the possibilities.

felipeespic commented 6 years ago

Yes, I am connected now actually.