About PA as attention

[TitaniumOne]

Thank you for your great work! I am really interested with your work!

The Lite modality is computed as the equation(7) ,but the code released is not fit the equation(7).

elif self.modality == 'Lite':
                input = input.view((-1, sample_len) + input.size()[-2:])
                PA = self.PA(input)
                RGB = input.view((-1, self.data_length, sample_len) + input.size()[-2:])[:,0,:,:,:]
                base_out = torch.cat((RGB, PA), 1)  # is not fit the equation(7)
                base_out = self.base_model(base_out)

So could you release the code about equation(7).

[incognite-lab]

You are wrong. The Eq. 7 refers to the PA module as attention, that is not used in the code. Instead the PA as motion cue is adopted in PA_modules.py

for i in range(self.n_length-1): d_i = nn.PairwiseDistance(p=2)(x[:,i,:,:], x[:,i+1,:,:]).unsqueeze(1) d = d_i if i == 0 else torch.cat((d, d_i), 1) PA = d.view(-1, 1*(self.n_length-1), h, w)

The code refers to Eq.10 where PA is mixed with RGB - see Fig. 3C