search

alexzhou907 / DDBM

138 stars 14 forks source link

About VP Schedule's Hyperparamter #2

Closed zytzrh closed 6 months ago

zytzrh commented 6 months ago

Hi @alexzhou907 ! It's really a nice work. It's thrilling to see it gain recognition and acceptance.

Upon reviewing both the paper and the source code, I've encountered a point of confusion regarding the implementation of the VP schedule's hyperparameter, specifically the constant drift term. The paper mentions that incorporating a constant drift term could enhance performance, which caught my attention.

image

In the code, however, it seems that there is no such implementation, or I might have overlook it. The most relevant section I found about VP schedule implementation is this line of code, which seems to still use the unbalanced diffusion bridge. Is there any point I miss?

https://github.com/alexzhou907/DDBM/blob/0d3cc96b852323e17250a7b5e29cc9e42e078bfa/ddbm/karras_diffusion.py#L21-L23

alexzhou907 commented 6 months ago

Hi there. Nice catch! I wrote this supplementary section a while ago and did updated experiments on VP afterwards but forgot to change this section. The finding should be aggressively increasing beta (as in the case of the default linear schedule in unconditional diffusion) degrades performance but slight increase of beta is ok. I chose beta_d = (beta_max - beta_min) = 2 and beta_min = 0.1 as default because it roughly matches the bridge noise level of the default VE bridge across time. I will update the paper soon. Thanks for pointing this out.

zytzrh commented 6 months ago

Thanks for your response!