weka511 / nlp

My experiments with Natural Language Processing. I've created a few programs to try out concepts.
GNU General Public License v3.0
1 stars 0 forks source link

Use Noise Contrastive Esimation (NCE) for word2vec #18

Closed weka511 closed 1 year ago

weka511 commented 1 year ago

Mikolov et al, Distributed Representations of Words and Phrases and their Compositionality, recommend creating negative samples, k for each central word, where k is 5-20 for small datasets, 2-5 for large.

weka511 commented 1 year ago

Fixed for word2vec2.py, still need to do word2vec.py

weka511 commented 1 year ago

Sort out terminology, NCE vs NEG

weka511 commented 1 year ago

Working