sunshangquan / Histoformer

[ECCV 2024] Histoformer: Restoring Images in Adverse Weather Conditions via Histogram Transformer
136 stars 10 forks source link

关于Dynamic-range Histogram Self-Attention中bin-wise histogram reshaping和frequency-wise histogram reshaping的代码实现 #3

Closed Zhouyang00 closed 3 months ago

Zhouyang00 commented 3 months ago
   尊敬的Shangquan Sun,您好!感谢您的出色工作,我对您提出的Dynamic-range Histogram Self-Attention十分感兴趣,将在最近的工作中进行引用您的这项工作。
   根据您提供的代码,及论文中的描述:To extract both global and local information, we define two types of reshaping, i.e., bin-wise histogram reshaping (BHR) and frequency-wise histogram reshaping (FHR). The first is to assign the number of bins equal to B and each bin contains HW/B elements, while the second is to set the frequency of each bin equal to B and the number of bins is HW/B. 
   想向您请教一下Bin的个数B在代码中如何实现的。代码中:hw = q.shape[-1] // self.factor 行显示您的Bin值设置为了 self.factor ,而 self.factor = num_heads,这与消融实验中Bin=36不太一致。
   此外,想请教您“第一种方法是将bin的个数设为B”和“第二种方法是将每个bin的频率设为B”在代码中是如何体现的呢。您方便的话,麻烦指出代码实现的行数或加以文字的描述。
   期待您的回复。如果上述我的个人理解存在对您代码理解有误,还请多包涵!
sunshangquan commented 3 months ago

Hi @Zhouyang00 , 有一点不一致的地方,我更新了一版arxiv,预计会在几天内放出,代码层面也会最近几天更新一下,代码体现在390-395行,bin和channel会打包在一起执行自注意力,bin为了方便赋值的是head的数量,36是第一层里面bin和channel的乘积,之后第二层是144,逐层递增,而实现bin的个数和频率分别为B的方式是第391行实现的,ifBox如果为True,以(factor hw)切分,反之则以(hw factor)切分

Zhouyang00 commented 3 months ago

感谢您的及时回复,很好地解决了我的疑问。祝您科研工作顺利!

sunshangquan commented 3 months ago

谢谢您对我们工作的关注,祝您工作生活顺利!