AttributeError: 'NoneType' object has no attribute 'tokenize'

649453932 / Bert-Chinese-Text-Classification-Pytorch

使用Bert，ERNIE，进行中文文本分类

MIT License

4.05k stars 902 forks source link

AttributeError: 'NoneType' object has no attribute 'tokenize' #198

Open Cgetier520990 opened 4 months ago

Cgetier520990 commented 4 months ago

Traceback (most recent call last): File "run.py", line 28, in train_data, dev_data, test_data = build_dataset(config) File "D:\PYTHON_PROGRAMME\Bert-Chinese-Text-Classification\utils.py", line 36, in build_dataset train = load_dataset(config.train_path, config.pad_size) File "D:\PYTHON_PROGRAMME\Bert-Chinese-Text-Classification\utils.py", line 20, in load_dataset token = config.tokenizer.tokenize(content) AttributeError: 'NoneType' object has no attribute 'tokenize' 这个报错怎么解决？

wangling6666 commented 4 months ago

我也出现这个错误，你解决了吗

Cgetier520990 commented 3 months ago

没有诶，你现在解决了吗？

CNMBE commented 3 months ago

@wangling6666 @Cgetier520990 我刚刚拉完代码也有这个问题，从huggingface把vocab.txt下载后放到模型文件夹就行