search

andrewgcodes / xlstm

my attempts at implementing various bits of Sepp Hochreiter's new xLSTM architecture
MIT License
111 stars 8 forks source link

it = torch.exp(torch.matmul(self.wi, x) + self.bi) #1

Open Tomcat099 opened 1 month ago

Tomcat099 commented 1 month ago 
    it = torch.exp(torch.matmul(self.wi, x) + self.bi)
    ft = torch.sigmoid(torch.matmul(self.wf, x) + self.bf)

    在it和fi的计算中，wi*x的过程中不应该是wi的T(即转置)和x相乘嘛？wf同理，在代码实现中好像没有体现出来，可以解答下嘛？