thanks for your brilliant work, i was reproducing the results on ROC unconditional generation, but i met with some problems
when roc story training, your code ends with eval loss ~0.055
1.but when i simplified your code and try to reproduce it, sqrt schedule converges at eval loss ~0.09, linear schedule achived ~0.07 which is still not good enough for my future work.
is that normal? since sqrt betas are much greater than linear
i've spent a lot of time debuging, and check every detail(my model have the same output, loss, grad and kept the same after optimizer one step as your model), but my code cannot converge at a better eval loss, can you give me any advise, I'd be really grateful.
thank you for your time, I could really need some help
thanks for your brilliant work, i was reproducing the results on ROC unconditional generation, but i met with some problems
when roc story training, your code ends with eval loss ~0.055 1.but when i simplified your code and try to reproduce it, sqrt schedule converges at eval loss ~0.09, linear schedule achived ~0.07 which is still not good enough for my future work.
2.here I noticed in sqrt schedule: betas = [0.01464131 0.00888909 0.00706818 ... 0.35722328 0.55561113 0.999 ] sqrt_one_minus_alphas_cumprod = [0.12100128 0.1529714 0.17407766 ... 0.99977265 0.99989897 0.9999999 ]
in linear schedule: betas = [5.00000000e-05 5.49774887e-05 5.99549775e-05 ... 9.99004502e-03, 9.99502251e-03 1.00000000e-02] sqrt_one_minus_alphas_cumprod = [0.00707107 0.01024572 0.01284225 ... 0.9999787 0.99997891 0.99997912]
is that normal? since sqrt betas are much greater than linear
i've spent a lot of time debuging, and check every detail(my model have the same output, loss, grad and kept the same after optimizer one step as your model), but my code cannot converge at a better eval loss, can you give me any advise, I'd be really grateful.
thank you for your time, I could really need some help