XLNet generates random word embeddings

amansrivastava17 / embedding-as-service

One-Stop Solution to encode sentence to fixed length vectors from various embedding techniques

MIT License

203 stars 29 forks source link

XLNet generates random word embeddings #45

Open maziyarpanahi opened 4 years ago

maziyarpanahi commented 4 years ago

Hi, Setting Dropout in production results in random embeddings. This has been discussed in the main repo if you search. To reproduce this, just run the same string more than once, each time you'll see different embeddings.

https://github.com/zihangdai/xlnet/pull/151#issuecomment-510442414

Saichethan commented 4 years ago

@maziyarpanahi is this solved??

maziyarpanahi commented 4 years ago

If the linked PR bing merged which sets Dropout to 0.0 then the random embeddings will be resolved as the authors of the XLNet suggested.

Saichethan commented 4 years ago

@maziyarpanahi so if I wanted to get XLNet embeddings now what should I do? Wait till they merge or is there any other way

chiragjn commented 4 years ago

Hey sorry, we kind of got busy with other things and totally forgot about this. I'll test out my PR today and get it merged. In the meanwhile if you can test out the patch in the PR and confirm that will also help