ML-GSAI / BFN-Solver

Official PyTorch implementation for "Unifying Bayesian Flow Networks and Diffusion Models through Stochastic Differential Equations"
32 stars 2 forks source link

Will BFN be effective on discrete dataset with large vocabulary? #1

Open LtECoD opened 3 months ago

LtECoD commented 3 months ago

Hi, The discrete BFN presented in the paper has demonstrated competitive performance on the text8 dataset. However, the vocabulary size of text8, which stands at a mere 27, is considerably limited for most NLP tasks. Have you experimented with training discrete BFN models on datasets with a larger vocabulary? Could you provide some insights into the model's architecture, settings of hyper parameters, and the performance achieved? Thanks!