In the toy example in Figure 2 in the paper, it's claimed that the angles (directions) of the neurons are crucial for storing semantic information. However, in the code for "forward_no_norm(self, x)", the following equations are used to process angular information:
x = x / xnorm
x = x / wnorm
x = x * 60
How is the angular information between x and the weights (filt) represented here? Did I miss something?
Hi,
Thank you for your excellent work.
In the toy example in Figure 2 in the paper, it's claimed that the angles (directions) of the neurons are crucial for storing semantic information. However, in the code for "forward_no_norm(self, x)", the following equations are used to process angular information:
x = x / xnorm x = x / wnorm x = x * 60
How is the angular information between x and the weights (filt) represented here? Did I miss something?