About time embedding algorithm algorithm

[sirius777coder]

Hi,

When I read your function: https://github.com/jasonkyuyim/se3_diffusion/blob/2c6405fb5a1c4a23354cc697c18ff478dff8546b/model/score_network.py#L35 i find your code is slightly different with the original DDPM in:timesteps = timesteps * max_positions. Is there any reason to add this line of code ?

[jasonkyuyim]

Yes, DDPM uses discrete time so timesteps is a element of positive integers, $\mathbb{Z}^n$. This fits in with the original sinusoidal embedding which was originally developed for embedding integer positions. However, our time steps are continuous over [0, 1] so we multiple it by the max_positions to emulate integers. This is important because if we don't then the time step embeddings will be nearly identical (assuming max_positions = 10000). Maybe max_positions=1 works equally but I haven't thought about it too much.

[sirius777coder]

Great! I'm curious about the choice of continuous time instead of discrete time. Does it correspond to the Stochastic Differential Equations (SDE) mentioned in your paper?

[jasonkyuyim]

Yes exactly

[sirius777coder]

Great! Thanks for your reply.