Open 787629504 opened 3 years ago
Hello, I want to ask you a question. In relative position embeddings , why should the number of channels be divided by 2(out_channels // 2)?
self.rel_h = nn.Parameter(torch.randn(out_channels // 2, 1, 1, kernel_size, 1), requires_grad=True) self.rel_w = nn.Parameter(torch.randn(out_channels // 2, 1, 1, 1, kernel_size), requires_grad=True)
The channels are divided into two parts, the first part contains relative height information(rel_h), and the other part contains relative width information(rel_w).
Hello, I want to ask you a question. In relative position embeddings , why should the number of channels be divided by 2(out_channels // 2)?
self.rel_h = nn.Parameter(torch.randn(out_channels // 2, 1, 1, kernel_size, 1), requires_grad=True) self.rel_w = nn.Parameter(torch.randn(out_channels // 2, 1, 1, 1, kernel_size), requires_grad=True)