search

Gutianpei / MID

[CVPR2022] Code for CVPR 2022 paper "Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion"
179 stars 28 forks source link

Clarification on Diffusion Step Embedding #16

Closed jxmmy7777 closed 1 year ago

jxmmy7777 commented 1 year ago

Hi, I have been exploring the codebase of your project and came across the implementation of diffusion step embedding in the MID/models/diffusion.py file. While reviewing the code, I noticed a particular operation regarding diffusion steps embedding that raised some questions and I would appreciate some clarification on the matter.

In the mentioned file, specifically in line 156 time_emb = torch.cat([beta, torch.sin(beta), torch.cos(beta)], dim=-1), it appears that beta, instead of the diffusion timesteps, is being treated as the time parameter for calculating the cosine and sine embeddings. I understand that beta and the diffusion timesteps have a one-to-one mapping, but their values range and change are different over the course of the diffusion process.

I was wondering if you could provide some insights or references explaining the rationale behind this operation. Thank you!

Gutianpei commented 1 year ago

Hello,

Thanks for the comment. As you said beta and diffusion timesteps have very different range, and as we trained model with beta scheduler, it'd make more sense to use beta as time_emb. We did not try different params on this topic so it might be optimized choice.