search

codertimo / BERT-pytorch

Google AI 2018 BERT pytorch implementation
Apache License 2.0
6.09k stars 1.29k forks source link

IndexError #88

Open LemonQC opened 3 years ago

LemonQC commented 3 years ago

I try to run according to md tutorial but image

Wushengyao commented 3 years ago

same issue

zxhjames commented 2 years ago

hey bro,do you have solutions?

syzhy113 commented 2 years ago

becauese there are some problems abouot spilt('\t').

shazifan commented 2 years ago

same issue, any solutions?

cheng940929 commented 2 years ago

check your corpus, it might not follow this rule in some lines: 'text \t text'

Chasingdreams6 commented 6 months ago

It likes the parser for '\t' works bad..

if you install bert_pytorch by pip, edit /opt/conda/lib/python3.8/site-packages/bert_pytorch/dataset/dataset.py line 23.

change 

self.lines = [line[:-1].split("\t")
                              for line in tqdm.tqdm(f, desc="Loading Dataset", total=corpus_lines)]

to 

self.lines = [line[:-1].split("\\t")
                              for line in tqdm.tqdm(f, desc="Loading Dataset", total=corpus_lines)]