search

zhouhaoyi / Informer2020

The GitHub repository for the paper "Informer" accepted by AAAI 2021.
Apache License 2.0
5.24k stars 1.1k forks source link

关于ProbAttention类中的_prob_QK函数 #615

Open wtdn2015 opened 4 months ago

wtdn2015 commented 4 months ago

想问一下该函数的返回值的含义是什么

PowerFour4 commented 2 months ago

return Q_K, M_top Q_K是论文中定义的kl散度近似算法的得分前n_top个Q计算出的注意力图 M_top 是Q_K注意力图来自哪些Q的索引