search

zjy-ucas / ChineseNER

A neural network model for Chinese named entity recognition
1.78k stars 569 forks source link

有个问题请教一下，word2vec中<UNK>的矩阵是怎么计算出来的呀，谢谢 #1

Open OYE93 opened 7 years ago

molyswu commented 7 years ago

python train.py Traceback (most recent call last): File "train.py", line 4, in from loader import load_data File "/home/molys/product/NER/chineseNER_ucas/ChineseNER/loader.py", line 32 SyntaxError: Non-ASCII character '\xef' in file /home/molys/product/NER/chineseNER_ucas/ChineseNER/loader.py on line 32, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details

syw2014 commented 7 years ago

@molyswu I found you had run it with python2 but this project was developed with python3, you should make some changes that it can be run with python2

syw2014 commented 7 years ago

@OYE93 This is OOV( out of vocabulary ) problem, and there are some methods to solve it like average , firtstly use the 'UNK' to replace the word which is OOV, then train the mdoel

OYE93 commented 7 years ago

Thanks, I used average.