jasonkyuyim
commented
2 weeks ago Hi, sorry for the late reply. Likelihoods can certainly be calculated for generative flow models. Since we assume independence between the discrete and continuous flows, it's the product of the two likelihood (see eq. 10). You can calculate the likelihood for the continuous flows using standard techniques in normalizing flows (see https://arxiv.org/pdf/2210.02747 and https://arxiv.org/abs/1806.07366). For discrete flows, it's a bit tricky if you're not assuming a left to right ordering because you have to compute the marginal over all possible unmasking orders. I'm not sure what your application is. For text, it should be fine.
Dear Esteemed Authors,
I am an avid follower of your work, particularly impressed by the paper 'Trans-Dimensional Generative Modeling via Jump Diffusion Models'. After implementing your methodologies, I have observed significant progress with my dataset, thanks to your profound insights. I am also optimistic that your paper 'Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design' will prove equally effective.
Currently, I am faced with the challenge of calculating the exact likelihood of my generative model. While I am familiar with calculating the exact likelihood for continuous state diffusion models based on stochastic differential equations (SDEs), I am uncertain if this approach extends to generative flow models, particularly when evaluating the likelihood of a specific sample in a complex sample space that includes both discrete and continuous elements. Is the likelihood in such cases well-defined?
Your contributions to the community have been invaluable, and any guidance you could offer on this matter would be greatly appreciated.
Warm regards