search

louaaron / Score-Entropy-Discrete-Diffusion

[ICML 2024 Best Paper] Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution (https://arxiv.org/abs/2310.16834)
https://aaronlou.com/blog/2024/discrete-diffusion/
MIT License
352 stars 33 forks source link

Missing Implementation for Calculating Likelihood Values as per Equation (10) #2

Open Monohydroxides opened 5 months ago

Monohydroxides commented 5 months ago

Hello, thank you for your outstanding work! In the code, I don't seem to find the implementation for calculating likelihood values as given by equation (10) in the paper. Could you please inform me about when you plan to upload it to the GitHub repository? Or perhaps I missed a part of the code that covers this calculation. Could you please assist me in identifying where the code for calculating likelihood values is located?

alexanderswerdlow commented 3 months ago

Following up on this, it seems the only perplexity metric in the repo is using the SEDD predictions as GT and measuring the perplexity of a pre-trained GPT-2 model, presumably to measure alignment with that model.

Would it be at all possible to provide the perplexity metric used in the paper?

Thanks!