how to do concatenative synthesis？

[Gchenchy]

Thank you for the great job!

Can I use libllsm2 to implement a simple concatenative synthesis program? If yes, can you give me some tips about how to do time stretching and how to do cross-fading between two recorded vowel audio samples?

Thank you very much!

[Sleepwalking]

Hello,

Thanks for your interest in libllsm2 and yes, libllsm2 was designed exactly for the purpose you mentioned.

Time stretching and cross-fading are both basically interpolating between frames. For each frame in the destination, you need to create a new frame (llsm_create_frame) and attach (llsm_container_attach) interpolated parameters to it. You can do it in either layer 0 or layer 1. In the case of cross-fading, doing it in layer 1 is recommended because you don't need to worry about two samples having different pitches.

However, libllsm2 does not provide the interpolator and you must implement your own. This is quite straight forward for fixed-dimensional parameters (e.g. the PSD vector in llsm_nmframe and the LLSM_FRAME_VTMAGN vector in layer 1). Interpolating between harmonic structures can be a bit tricky because they can have different numbers of harmonics. What I usually do is to pad them with zeros to even out the difference. Another caveat is when you interpolate between a voiced and an unvoiced frame. Just carefully write down the code to treat the amplitude of voiced component in the unvoiced frame as zero.

[Gchenchy]

@Sleepwalking Thank you very much for the solution. Implementing it seems can be a bit tricky. Do you have any plans to open source some relevant test examples ?

[Sleepwalking]

https://github.com/Sleepwalking/libllsm2/commit/15ec4608a88ca62d4c46d96fc63f6e195866c768

[Gchenchy]

@Sleepwalking Nice. Thank you very much!