fastnlp / fastNLP

fastNLP: A Modularized and Extensible NLP Framework. Currently still in incubation.
https://gitee.com/fastnlp/fastNLP
Apache License 2.0
3.05k stars 451 forks source link

StaticEmbedding param vocab: Vocabulary. 若该项为None则会读取所有的embedding。 #346

Open el-psy opened 3 years ago

el-psy commented 3 years ago

from fastNLP.embeddings import StaticEmbedding word2vec_embed=StaticEmbedding(None, model_dir_or_name='cn-char-fastnlp-100d')

Traceback (most recent call last): File "d:\data\node_with_att_bert_data_tunnel.py", line 175, in word2vec_embed=StaticEmbedding(None, model_dir_or_name='cn-char-fastnlp-100d') File "E:\ProgramData\Anaconda3\lib\site-packages\fastNLP\embeddings\static_embedding.py", line 96, in init super(StaticEmbedding, self).init(vocab, word_dropout=word_dropout, dropout=dropout) File "E:\ProgramData\Anaconda3\lib\site-packages\fastNLP\embeddings\embedding.py", line 127, in init if vocab.rebuild: AttributeError: 'NoneType' object has no attribute 'rebuild'

请问可否有其他的方法将embedding的所有字加载,或者直接生成对应的Vocabulary?

yhcc commented 3 years ago

抱歉,这是一个错误的文档说明。 目前只能通过自己写一下读取embedding的方式,我们最近会开发一下直接读取整个embedding的代码。