search

mmorise / World

A high-quality speech analysis, manipulation and synthesis system
http://www.kisc.meiji.ac.jp/~mmorise/world/english
Other
1.15k stars 249 forks source link

Cpu efficiency is very low,when using streaming world #117

Closed zhouchunliang closed 3 years ago

zhouchunliang commented 3 years ago

cpu env config CPU(s): 72 On-line CPU(s) list: 0-71 Thread(s) per core: 2

synthesisrealtime world: 30 threads：every threads cpu efficiency is 100%， cpu efficiency is very high 60 threads：every threads cpu efficiency is 70%，cpu efficiency is very low 140 threads：every threads cpu efficiency is 65%，cpu efficiency is very low synthesis world: 30 threads：every threads cpu efficiency is 100%， cpu efficiency is very high 60 threads：every threads cpu efficiency is 100%，cpu efficiency is very high 140 threads：every threads cpu efficiency is100%，cpu efficiency is very high

Cpu efficiency is very low，when multithread use synthesisrealtime world（more than a certain number of concurrency） Cpu efficiency still keep very high，when multithread use synthesis world（more than a certain number of concurrency）

Under multithreading, synthesisrealtime world performance is not as good as synthesis world(such as response time, real-time rate etc)，The main reason is that synthesisrealtime world cannot make full use of cpu resources；How to make streaming synthesisrealtime world response time faster than non-streaming synthesis world?

mmorise commented 3 years ago

Thank you for your comment.

It is difficult to identify the problem based on only the given information completely. Generally, real-time synthesis is sequentially carried out because the past speech parameter affects the current waveform. When you have generated speech parameters sequentially, the CPU efficiency may decrease. In this case, if the computational cost for generating speech parameters is larger than that of speech waveform, the CPU efficiency decreases.

zhouchunliang commented 3 years ago

Thank you for your comment.

It is difficult to identify the problem based on only the given information completely. Generally, real-time synthesis is sequentially carried out because the past speech parameter affects the current waveform. When you have generated speech parameters sequentially, the CPU efficiency may decrease. In this case, if the computational cost for generating speech parameters is larger than that of speech waveform, the CPU efficiency decreases. The parameter conversion is completed at one time. The only difference is that the vocoder is changed from non-streaming to streaming in TTS system. After this operation, the CPU utilization rate and performance decreases. Is there any internal multi-thread processing in synthesisrealtime world?

mmorise commented 3 years ago

There is no multi-thread processing in the real-time synthesis. It is possible to use multi-thread processing, but it would be difficult for waveform generation of one frame to use the CPU by multi-thread processing effectively. Furthermore, since the waveform generation of the current frame requires the information of the past speech parameters, I think multi-thread processing is unsuitable for real-time synthesis.

zhouchunliang commented 3 years ago

There are 300 frames of data, addParameters and synthesis2 are used once every 5 frames of data, 60 executions completed, finally，there is a crash; All 300 frames are added at the same time, there is no crash；How to use addParameters and synthesis2 add several frames at the same time，not all frames are added at the same time?

mmorise commented 3 years ago

I'm afraid that it is difficult to solve this problem without the sample code for replicating the error. The test code (test.cpp) includes both examples you suggested. If you observed such an error, I think that your code includes an error.