Subtracting mean embeddings

vyraun / Half-Size

Code for "Effective Dimensionality Reduction for Word Embeddings".

128 stars 24 forks source link

Subtracting mean embeddings #2

Open bwang482 opened 4 years ago

bwang482 commented 4 years ago

Are you sure this line is correct? X_train = X_train - np.mean(X_train)

np.mean(X_train) gives a single value. Shouldn't it be np.mean(X_train, 0) ???

GuilhermeZaniniMoreira commented 4 years ago

I am gettings this error: ValueError: operands could not be broadcast together with shapes (237191,) (300,) There are 237.191 words with embeddins space equals to 300. How did you solve that?

iR00i commented 3 years ago

Shouldn't it be np.mean(X_train, 0) ???

If you go back to the original paper that proposed the "Post-Processing Algorithm" (All-but-the-Top: Simple and Effective Postprocessing for Word Representations), the authors outline computing the mean to be the following:

So i imagine the resulting mean should be a scaler computed from the entire matrix.