search

shivammehta25 / Matcha-TTS

[ICASSP 2024] 🍵 Matcha-TTS: A fast TTS architecture with conditional flow matching
https://shivammehta25.github.io/Matcha-TTS/
MIT License
718 stars 88 forks source link

Question about OT-CFM #112

Open umiuri1105 opened 20 hours ago

umiuri1105 commented 20 hours ago

Thank you for your great work. I have a question about Flow Matching. The Matcha-TTS paper states that OT-CFM is used, but where is the OT-CFM program implemented in this GitHub program? It looks like the program implements general CFM (I-CFM).

shivammehta25 commented 11 hours ago

Hello thank you for your interest in our work.

Matcha-TTS, uses the conditional Optimal Transport introduced in Flow Matching for Generative Modeling . In section 4 Example II, the equation of the flow is described in more detail with a disclaimer.

Lastly, we note that although the conditional flow is optimal, this by no means imply that the marginal VF is an optimal transport solution. Nevertheless, we expect the marginal vector field to remain relatively simple.

We use the same definition and formulation.

I am sorry, I am not familiar with I-CFM, could you please provide a reference to it so I can take a look at it?

umiuri1105 commented 11 hours ago

Thank you for your response. I-CFM is explained in Improving and Generalizing Flow-Based Generative Models with Minibatch Optimal Transport. OT-CFM is also discussed in this paper. When implementing OT-CFM, is it necessary to use the library "pot"?

shivammehta25 commented 11 hours ago

I-CFM is explained in Improving and Generalizing Flow-Based Generative Models with Minibatch Optimal Transport.

Ah ofcourse I have read this article, I just couldn't recall that they call it I-CFM. I think they have their implementation open sourced. And they seem to be using pot and from what it looks, it just provides a sample plan off the shelf.