Closed Gliterament closed 2 years ago
The element of the array depends on what the parameters sent to it. By default, it retrieves the second to last hidden layer sentence embeddings. This has to do with it generally being a better representation than the last layer. The average depends on the axis sent to it. Generally the dimension is set to 1.
In the function 'extract_embeddings', it says that the function returns a numpy array, I'm wondering what are the elements of the array and I was thinking that the 'pooled' might refer to a float that is the average of (batch_size, sequence_length, hidden_size) in a torch tensor. Please help me figure this out, it's very important to me.