search

DeepGraphLearning / SiamDiff

Code for Pre-training Protein Encoder via Siamese Sequence-Structure Diffusion Trajectory Prediction (https://arxiv.org/abs/2301.12068)
38 stars 5 forks source link

Question about the time encoding #2

Open violet-sto opened 10 months ago

violet-sto commented 10 months ago

Hi, I have a question about the time encoding. As DiffPreT is trained like a diffusion model, does the backbone network GearNet explicitly encode the time step?

Oxer11 commented 9 months ago

Hi, thanks for the question. This is essentially a difference bewteen diffusion models for generation and pre-training. In pre-training, we do not explicitly encode the time step to accomordate different encoder architectures and keep the consistency between pre-training and fine-tuning, as no noise is introduced during fine-tuning. Instead, we can implicitly encode the noise level (time step) by using a perturbed distance encoder. https://github.com/DeepGraphLearning/SiamDiff/blob/2bc9682fbdfde76b75a3cc419fe4ad6bb7a7d85b/siamdiff/task.py#L208