solve a bug which will fail to build vocab

stanleylsx / entity_extractor_by_ner

基于Tensorflow2.3开发的NER模型，都是CRF范式，包含Bilstm(IDCNN)-CRF、Bert-Bilstm(IDCNN)-CRF、Bert-CRF，可微调预训练模型，可对抗学习，用于命名实体识别，配置后可直接运行。

390 stars 73 forks source link

Closed weiambt closed 1 week ago

weiambt commented 5 months ago

The problem is in the file engines/data.py line 108.

old:

tokens = [tokens for token in tokens if token if token not in [' ', '']]

modified:

tokens = [token for token in tokens if token if token not in [' ', '']]