Setting global vector to some values

[earthpimp]

In Earthformer, the number of global vectors is set to 8 which improves the prediction quality a lot without adding much computational cost. I wonder why is it set to 0 here. Is there any specific reason?

[gaozhihan]

Thank you for your question. The implementation of global attention in practice introduced a slightly more computational overhead. We set the number of global vectors to 0 for faster trials. While using global vectors may further increase the performance, we didn't give a try due to the high costs of diffusion model training and inference.

[earthpimp]

I have encountered some problems again. I found that the model is producing varying outputs for the same input (I am using the original sampler from ddpm). Is that due to the use of xformer? Or it is just a kind of property for conditional diffusion? Sorry for bothering you but I am quite new to diffusion model, so I will be appreciated if you can offer some help.

[gaozhihan]

Thank you for your question. This is an intrinsic property of a probabilistic forecasting model. As introduced in Section 2.1 in our paper, the model parameterizes $p\theta(x|y)$ instead of a deterministic mapping (a $\delta$ distribution) from $y$ to $x$, to represent the inherent uncertainty of the system. Therefore, for a given input $y$, various plausible futures $x$ can be generated by sampling from $p\theta(x|y)$.