Open shainaraza opened 4 years ago
The first dimension are the different sentences, the second dimension are the embeddings for each token / sub-token.
For dimension reduction, you would need a dimension reduction technique like PCA or LSA.
As BERT uses contextualized word embeddings, i.e., the word embeddings depend on the complete context. If you only pass individual words, you would loose this feature from BERT
The first dimension are the different sentences, the second dimension are the embeddings for each token / sub-token.
For dimension reduction, you would need a dimension reduction technique like PCA or LSA.
As BERT uses contextualized word embeddings, i.e., the word embeddings depend on the complete context. If you only pass individual words, you would loose this feature from BERT
superb thanks
Thanks for providing information, for the word embedding, I use following code as given in one of the examples
it gives me an array of array something like this [[-0.7301325 -0.3941787 2.3462217 ... 0.10826997 -0.27091306 -0.08751018] [-0.5559533 -0.5525051 2.3269138 ... -0.50415146 -0.54448223 -0.11626344] [-0.4228519 -0.3109586 2.918267 ... 0.03306676 -0.6508648 0.09592707] [-0. -0. 0. ... -0. -0. -0. ]]
for my understanding, what are the subarrays? Also I have another question, can I reduce the embedding size to 10,20 or 50.
Also, If I have a long sequence of words, should I pass the whole sequence as one sentence (which i did) or tokenize each word and pass as list of elements within encode to get word embeddings.
thanks you